ArcPy - Scripting with ArcPy

ENV 859 - Geospatial Data Analytics   |   Fall 2023   |   Instructor: John Fay  

Introduction

In our last session, we started a scripting project to convert raw ARGOS data into a GIS feature class. We got as far as being able to extract coordinates from the data, but to proceed we need to pause and explore ArcPy - ESRI’s Python package to perform spatial analysis in a coding environment.

Here we will take pause from our project and dive into ArcPy, using our scripting project as context for targeted learning what ArcPy can do and how to go about learning how to use it.

LAB REQUIREMENTS AND MATERIALS To run these exercises, we’ll need:

  1. A machine on which ArcGIS Pro & VS Code installed,
  2. The ArcPyDemo1 workspace (located here) downloaded and unpacked on your V: drive.

Learning objectives

Topic On completion, you should be able to …
1. Running ArcGIS Tools from Python ♦ Access the Python command prompt in ArcGIS Pro
♦ Run a geoprocessing tool from the Python command prompt in ArcGIS Pro
♦ Run a geoprocessing tool from within the Spyder IDE
♦ Locate the proper Python syntax for a tool within the ArcGIS Pro help
♦ Identify which parameters are required and which are optional for a tool
♦ Explain why adding empty strings as tool parameter is sometimes needed
♦ Explain how spatial datasets are referenced in geoprocessing tools (i.e. as strings)
♦ Set the tool output as a variable, and use that output in a subsequent tool
2. ArcPy Functions ♦ Describe what a Python function is and the form it takes in Python
♦ Explain what a parameter is in the context of a function
♦ Access the complete list of ArcPy functions added when arcpy is imported
Learn & execute an ArcPy function by using ESRI’s help documentation
3. ArcPy Classes ♦ Describe how a Python class resembles a blueprint
♦ Explain the terms “properties” and “methods” in reference to class
♦ Access the complete list of ArcPy classes added when arcpy is imported
♦ Identify which properties and methods are associated with a given class
♦ Implement an ArcPy class in Python, using the help as a guide
4. ArcPy Environment Settings in Python ♦ Get and set ArcGIS environment settings using arcpy.env
♦ Enable your model to overwrite existing output with arcpy.env.overwriteOutput
5. Using parameters in your script ♦ Enable user input in your script with sys.argv[]
♦ Enable user input in your script with getParameterAsText()
♦ Explain the differences between these two methods
6. Creating ArcGIS Script tools &
  Adding user messages
♦ Create a script based tool in ArcGIS Pro
♦ Add appropriate parameters to your ArcGIS Pro script tool
♦ Explain how script tool parameters relate to script code parameters
♦ Generate messages in your code that are sent back to ArcGIS Pro
♦ Use setParameterAsText() to return defined output in your script
7. Describing data with ArcPy ♦ Create a describe object from a specified dataset
♦ Accessing dataset properties via the describe object
♦ List all the properties associate with a dataset
8. Data access using cursors ♦ Describe what cursors are in the context of accessing data tables
♦ Use a search cursor to retrieve features from a feature class or table
♦ Describe how a search cursor differs from an update or insert cursor
9. Working with geometry objects ♦ Access a feature class’ geometry object via a search cursor
♦ Get and set the properties of a geometry object
♦ Create a new Point, Polyline, or Polygon object via ArcPy
♦ Explain the difference between a shape object and geometry object

1. Running ArcGIS geoprocessing tools in Python

One of the key features of ArcPy is its ability to run any ArcGIS tool from Python. Here we examine the fundamentals of running ArcGIS tools from Python. We begin by diving in - first by exploring how tools are run via a Python command prompt in ArcGIS Pro, and then via the Spyder IDE. From there, we review a few approaches for teaching ourselves more about how to incorporate ArcGIS tools into out Python scripts.

A key objective here is to understand that all geoprocessing tools available within ArcGIS Pro can be run from a Python coding environment, and the key to running a tool is understanding the syntax that that tool requires

:point_right: More detailed info on running geoprocessing tools from Python is found here, in ESRI’s on-line help documentation.

» Exercise 1a: Executing a tool using Python within ArcGIS Pro

You may have, at one point or another, noticed that you can open a Python window from within ArcGIS. While this Python prompt is not suited for writing scripts (debugging is a challenge), it is useful for running specific Python statements. Here, we’ll explore how ArcGIS commands are executed in Python from the ArcGIS Python script.

  1. Unzip the ArcPyDemo1.zip file to your V: drive and open the project. This workspace includes a sample dataset of the San Diego area including roads, a set of study quads, areas below 250 m in elevation and areas less than 40% slope.

  2. Open up the Python command line window in ArcPro: Analysis Tab>Python button.

  3. At the Python prompt, type the following command to buffer the roads feature class 500 meters.

    arcpy.Buffer_analysis('majorrds','rdsBuff_500m','500 meters')
    

Notice how the intellitype feature of ArcGIS helps you format the command and how some basic information on the tool is provided at the right hand side.

Also notice that there is no need, when running Python from within ArcGIS, to import ArcPy; it is imported automatically for obvious reasons. The Python prompt, however, works exactly like the console in Spyder.

  1. Run the command and examine the output.

  2. Use the key on your keyboard at the Python prompt to scroll up to the Python statement you just ran.
    Edit the command so that it now appears as below:

    arcpy.Buffer_analysis("majorrds","rdsBuff_500m","500 meters","","","ALL")
    

Here we’ve altered the command to dissolve all the output features. This option to dissolve is the 6th argument in the tool’s syntax. The first 3 are required, but we skip the 4th and 5th ones (“line_side” and “line_end_type”) by supplying empty strings, to get to the 6th. Arguments are supplied in a specific order; we cannot skip arguments to get to a specific one. Instead we can supply empty strings to assign default values. We’ll talk more about command parameters shortly.

  1. The outputs of the above commands are added to the map, and as part of our map, we can use these outputs in later Python commands just as we did the original feature classes. Alternatively, we can assign the outputs to Python variables and use them in subsequent commands:

    bufferedFeatures = arcpy.Buffer_analysis("majorrds","rdsBuff_500m","500 meters")
    
  2. Where are we writing all these outputs? Check the environment settings to see whether that’s determining where they are saved. Turns out that we have to be explicit about path name if we want to control where the outputs go. We can do that with variables:

    outFC = "V:\\ArcPyDemo1\\Scratch\\clippedBuffers.shp"   
    arcpy.Clip_analysis(bufferedFeatures,"elevlt250",outFC) 
    

The Python window in ArcGIS is a good way to dive into using tools in Python and should shed a bit more light on how to go about running tools via Python. The intellitype feature and the brief help were useful in structuring our syntax, but there are other places to go to get a more thorough explanation of how specific tools are used in Python.

Other “cheats” to see what the Python syntax of a given tool include:

  • Running a geoprocessing tool in ArcGIS Pro, then navigating to your Geoprocessing History (from the Catalog Pane), right-clicking the process, and selecting “Copy Python Command” or the other means for looking at the code in Python.
  • Creating a geoprocessing model in the ArcGIS Pro model builder and then exporting the model to a script.

Results on both these methods may vary, however, as some tools convert to script better than others.

» Exercise 1b: Executing a tool using Python within an IDE

Here we’ll do a task similar to what we did in the previous example, but from entirely within VSCode. We’ll create a script that selects roads of a certain class value and buffers them a given distance.

  1. Start VSCode and open the Scripts folder (found within the ArcPyDemo1 workspace), trusting the folder.

  2. Create a new Python script, saving it as SelectExample.py (or whatever you want).

  3. View the help on the Select (Analysis) tool (here’s a link). Scroll down to the end of the page to see an example of how the tool is used in Python. Let’s start by copying what ESRI gives and tailoring it to our example. Copy the Select Example 2 (stand-alone Python script) to the clipboard and paste it into your Python script:

    # Name: Select_Example2.py
    # Description: Select roads of Class 4 from major roads tin the gnatcatcher habitat study area
       
    # Import system modules
    import arcpy
       
    # Set workspace
    arcpy.env.workspace = "C:/data"
       
    # Set local variables
    in_features = "majorrds.shp"
    out_feature_class = "C:/output/majorrdsClass4.shp"
    where_clause = '"CLASS" = \'4\''
       
    # Execute Select
    arcpy.Select_analysis(in_features, out_feature_class, where_clause)
    
  4. Change arc.env.workspace to point to the “SanDiego “folder within your data folder: V:/ArcPyDemo1/Data/SanDiego. (This directory, coincidentally also contains a shapefile called majorrds.shp!)

  5. Change the out_feature_class variable so that the majorrdsClass4.shp shapefile is created in your scratch folder.

  6. Save and run the Python script. Then check your scratch folder to see whether the file was created. (A link to working code is here)

  7. Now, let’s add another set of commands to buffer our “class 4” roads.

    buffRoads = "V:/ArcPyDemo1/scratch/BufferedRoads.shp"
    buffDist = "500 meters"
    arcpy.Buffer_analysis(out_feature_class,buffRoads,buffDist,'','','ALL')
    

    Something things to keep in mind: out_feature_class is actually the input for this tool; buffRoads is the tool output. Also, as in the preceding example, we have two “placeholder” inputs (the empty strings) for the two optional parameters, so that we can assign “ALL” as the dissolve option.

  8. Run the script and examine your scratch folder. Or better yet, open the outputs in ArcGIS. (Working code link.)

    Note you’ll have to manually delete the majorrdsClass4.shp file each time you run this. Well see how to fix this soon!

Key takeaways for running geoprocessing tools in Python:

  • You need to import arcpy to gain access to the geoprocessing tools.
  • Implementing a geoprocessing tool is mostly a matter of using the correct syntax. Looking up the tool in the ArcGIS Pro help reveals what the proper syntax is along with code samples.
  • The order and position in which parameters are specified is important. If you want to set the “dissolve_type” of the buffer command, and its the 6th parameter listed in the tool’s documentation, you need to be sure enter it as the 6th parameter, using empty strings as placeholder for preceding parameters, if not used.
  • Datasets are referenced to by their path on the machine. This goes for tool inputs and outputs. [Though if you are using Python within an ArcGIS session, the layers loaded into your current map are available as tool inputs…]

» Challenge!

  • How would you alter the above script so that it allowed the user to specify the buffer distance?
    What about the output file?
  • Could you modify the above script so that it buffered the selected roads three different distances - say 500, 1000, and 1500 meters - creating a separate output shapefile for each?

2. ArcPy functions

When we imported the math module into our Python coding environment in a previous lesson, we saw that it allowed us to do things we couldn’t have easily done beforehand: we could compute logs (log(7), square roots (sqrt(25)), cosines (cos(3.14)), etc. Stating this differently, importing the math module added new functions to our coding environment. The same is true when we import the arcpy package.

A Python function executes a specific task and often requires parameters. All calls to a function in Python are followed by parentheses, which is where we specify those parameters. If no parameters are required, we still include the parentheses but don’t insert anything between them. The order in which the parameters are supplied is important in executing the function correctly.

You may have already guessed this, but yes: all ArcGIS geoprocessing tools are accessible as arcpy functions! However, importing the arcpy package gives us additional functions that are not included as geoprocessing tools in ArcGIS Pro! We’ll take a look at this in the exercise below.

More detailed info on ArcPy Functions is found here:
http://pro.arcgis.com/en/pro-app/arcpy/geoprocessing_and_python/using-functions-in-python.htm

A complete list of ArcPy functions is available here:
http://pro.arcgis.com/en/pro-app/arcpy/functions/alphabetical-list-of-arcpy-functions.htm

» Exercise 2: Functions

This exercise offers some example applications of ArcPy functions…

→ First we’ll investigate the Exists function:

  • Navigate to the online help page for the Exists function (link).

  • Use the ArcPy Exists function to return a Boolean indicating whether the V:\ArcPyDemo1\data\SanDiego\climate.shp file exists.

    Reveal code arcpy.Exists('W:\\ArcPyDemo1\\data\\SanDiego\\climate.shp')

Now we’ll explore the ListFields function (link):

  • Use the ArcPy ListFields function to generate a list of all the fields in the V:\ArcPyDemo1\data\SanDiego\climate.shp table, saving the output as a variable named “myFields”.

  • What is the Python data type of the myFields variable you just created?

  • How many fields are in the table?

  • Extract the first item in this list into a new variable named “myField”.

  • What is the Python data type of this “myField” object?

    Reveal code to get answers myFields = arcpy.ListFields('V:\\ArcPyDemo1\\data\\SanDiego\\climate.shp')
    type(myFields)
    len(myFields)
    myField = myFields[0]
    type(myField)

Challenge!

  • Can you generate a list of just the “String” type fields in the V:\ArcPyDemo1\data\SanDiego\climate.shp dataset?

    Reveal answer stringFields=arcpy.ListFields('V:\\ArcPyDemo1\\data\\SanDiego\\climate.shp','','String')

3. ArcPy classes

In the exercise above, the ListFields function returns a list. Each item in this list is a field object; more precisely, each item is an ArcPy object belonging to the field class.

A class is roughly the same as what we’ve been calling “data type”. All scripting objects (that is, anything we can assign a variable to in Python) belongs to a class. And the class to which it belongs determines what properties can be assigned to the variable, and what we can do with the variable, or its methods.

To use a familiar example, in the code myName = "John", the variable myName become an instance of the Python string class. and thus inherits all the properties (its length, whether it’s alphanumeric, etc.) and all the methods (split, index, upper,…) defined by the Python string class.

ArcPy has many, many of its own classes. Some examples are point, field, extent, and value table (e.g. for use in reclassifications).

More detailed info on ArcPy Classes is found here:
https://pro.arcgis.com/en/pro-app/arcpy/geoprocessing_and_python/how-to-use-classes-in-python.htm

A complete list of ArcPy classes is available here:
https://pro.arcgis.com/en/pro-app/arcpy/classes/alphabetical-list-of-arcpy-classes.htm

An example: the Point class

As in all of Python, classes are used in ArcPy to create objects that have a specific set of properties and methods. For instance, the following statement creates a new point object, meaning a script variable assigned to the ArcPy “Point” class:

myPoint = arcpy.Point()

The myPoint variable is now said to be a “Point object”, and it therefore assumes the defined properties and methods of any ArcPy point, listed here: http://pro.arcgis.com/en/pro-app/arcpy/classes/point.htm . What then are these properties and methods??

♦ Properties

An object’s properties define or describe an object. For example, our point object has properties that define its X, Y, and Z coordinates, as well as a measure and ID property.

We can set our point’s X and Y coordinates as follows:

myPoint.X = -117.1436447
myPoint.Y = 32.8655

Some properties can be both read and changed within a script: a point object can be moved by changing its X and Y property values. However, other properties can only be read. An example would be a raster object’s “format” property which indicates what kind of raster it is (e.g. GRID, TIFF, etc.), and can only be changed by converting the raster, not simply by changing its property value.

♦ Methods

An object’s methods are all the actions that you can perform using the object. For example, a point object has a method called “within”:

isWithin = myPoint.within(someOtherShapeObject)

This method allows you, via a script, to determine whether a point falls within another [point, line, or polygon] object.

Classes, properties, and methods are by no means unique to ArcPy. In fact, whenever we assign a variable in Python, is becomes the member of a class – which class is discovered using the type() function. For example, if we assign X = "Python is fun", X becomes a member of the Python string class, and it has properties (e.g. a length, whether it’s upper case) and methods (capitalize, replace) of its own.

» Exercise 3: Classes

  • Create an ArcPy point object:

    myPoint = arcpy.Point()
    
  • Assign values to the X, Y and ID properties of the point object:

    myPoint.X = 6266956
    myPoint.Y = 1876781
    myPoint.ID = 1
    
  • Extract the first polygon from the San_Diego study_quads.shp feature class (don’t worry how this is done quite yet…)

    myPoly = arcpy.SearchCursor("V:\\ArcPyDemo1\\data\\SanDiego\\study_quads.shp").next().Shape
    
  • Use the Point’s within method to see whether our point is within the polygon.

    print (myPoint.within(myPoly))
    
  • List some properties of the myPoly polygon object.

    • What is it’s area?
    • How many points is it made of?

4. Using ArcPy environment settings in Python

More info on Environment settings:
http://pro.arcgis.com/en/pro-app/arcpy/geoprocessing_and_python/using-environment-settings.htm

Environment settings, such as the current and scratch workspace locations, are set as properties of the ArcPy env class. Setting these can be useful and, in some cases, essential. An example of this is seen in Exercise 1b above where we set the “workspace” environment variable (arcpy.env.workspace = "C:/data"); in doing so, we no longer have to supply full paths to the “majorrds.shp” dataset as ArcPy now assumes the dataset exists in the current workspace.

Environment variables work just as they do in the ArcGIS Pro application: once set, other procedures can refer to those values which facilitates coding.

» Exercise 4: Getting and setting environment values

  • Import ArcPy and set the current workspace to “V:\ArcPyDemo1\data\SanDiego”

    import arcpy
    arcpy.env.workspace = "V:/ArcPyDemo1/data/SanDiego"
    
  • We can set the outputCoordinateSystem value to force our processes to create output in a specific coordinate system:

    arcpy.env.outputCoordinateSystem = arcpy.SpatialReference("WGS 1984")
    
  • Another environment setting allows us to overwrite the output, if it exists:

    arcpy.env.overwriteOutput = True
    
  • Now, let’s re-run the remaining code we did in Exercise 1b…

    # Set local variables
    in_features = "majorrds.shp"
    out_feature_class = "majorrdsClass4.shp"
    where_clause = '"CLASS" = \'4\''
      
    # Execute Select
    arcpy.Select_analysis(in_features, out_feature_class, where_clause)
    

    Where did the output go?
    What projection are the files in?


5. Using parameters in your script

If we wanted to re-run scripts with slightly different values, e.g. buffer a point 200 meters instead of 100 meters, we could always open our script up and edit it. A more elegant way of doing this, however, is to allow the item we want changed to be set outside of our script every time it is run - in other words, by setting variables as parameters rather than hard coded values.

We saw in previous exercises that Python’s sys.argv[] object can handle this task, but ArcPy provides additional means for interacting with values and objects passed into and out of your code. Here, we discuss the two ArcPy functions that are most typically used for this purpose: getParameterAsText() and setParameterAsText().

More info on using parameters in ArcPy scripts is found here:
https://pro.arcgis.com/en/pro-app/arcpy/functions/copyparameter.htm

In these exercises, we examine how sys.argv[] and getParamterAsText() work similarly.

» Exercise 5a: Revisiting sys.argv

The sys.argv object is, in computer jargon, an argument vector - which just means it’s a list containing arguments. The values of this list are supplied when the user runs the script, either from within VSCode or ArcGIS. (More on that in a bit…)

  • Write a new Python script using the following lines of code to see how sys.argv works. (Or you can load the Exercise5a.py script into your VSCode session…)

    import sys
    className = sys.argv[1]
    distance = sys.argv[2]
    A = sys.argv[3]
    B = sys.argv[4]
    
  • Now we’ll run the script in VSCode. However, to do that, we need to create a special file (“launch.json”) that includes the arguments we want to pass into the script so that we can debug the script:

    • With the Exercise5a.py script active, click the debug icon from the left hand action bar in VSCode.
    • In the “Run and Debug” window you should see a link to “create a launch.json file”. Click that link.
    • Select the option for “Python File” in the dialog box that appears.
    • You are then presented with your launch.json file in VSCode’s text editor. Add a comma ad the end of line 13 ("justMyCode": true,") and then hit Enter to add a new blank line just below it.
    • In this new line add the following text: "args": ["ENV859", "100 meters", "100", "17.6"]
  • Navigate back to the Exercise5a.py script and run it in debug mode.

  • After the script is run, we can examine the variable values at VSCode’s Python console:

    print(distance)
    
  • Print the sum of A and B. What does the result say about the data types passed into your script via the sys.argv technique?

  • The sys.argv itself is a list. What is the first element in this list?

» Exercise 5b: Using arcpy.getParameterAsText()

ArcPy’s GetParameterAsText() function works just like sys.argv. With one key exception: GetParameterAsText does not include the script’s name as its first element, so user inputs begin at position “0” in the list.

  • Open the script Exercise5b.py in VSCode and run it as you ran Exercise5a.py. Note that it replaces sys.argv[] with arcpy.GetParameterAsText(); the rest of the script is virtually the same.
  • Run the script as you did the previous one (in debug mode), and enter the same command line options as before.
  • Results should be the same!

Why use one over the other? sys.argv has limitations on the number of characters it can accept. GetParameterAsText() has no character limit. So it would seem the latter is the better choice. However, as we’ll see later, it’s sometimes useful to know the name of the script that stored as sys.argv[0] – and sys.argv does not require an expensive ArcGIS license, so it’s useful to know both are at your disposal.


6. Creating ArcGIS scripts tools & User messages

Now that we have a way of specifying user input in our scripts, we can link our code back to ArcGIS Pro as script tools, i.e., so that they can be run as tools in a geoprocessing model. We can also set our scripts to send messages back to the ArcGIS Status window – as opposed to “printing” messages to an interactive window that wouldn’t appear when run from ArcGIS.

» Exercise 6: Running scripts from ArcGIS Pro

  • Create a new Python script named Exercise6.pywith the following lines of code:

    import arcpy
      
    #Set variable names to the 4 user inputs
    className = arcpy.GetParameterAsText(0)
    distance = arcpy.GetParameterAsText(1)
    A = arcpy.GetParameterAsText(2)
    B = arcpy.GetParameterAsText(3)
      
    #Message results back to the status window when run
    arcpy.AddMessage(f"The class name is {className}.")
    arcpy.AddMessage(f"The distance is {distance}.")
    arcpy.AddMessage(f"{A} + {B} = {A+B}?!")
    
  • Run your script (using ENV859 "100 meters" 100 17.6 as inputs) to ensure it has no errors. Nothing should be printed to your interactive window. Why?

  • Open your ArcPyDemo1.arpx ArcGIS Pro project.

    :exclamation: Be sure that your Python environment is the default arcgispro-py3 or you may run into some errors.

  • Right click on your toolbox and select New -> Script to begin adding your script as a script tool.

  • Give your script a name, label, and then link it to your Exercise6.py script file.

  • Click on the Parameters tab to add our 4 script arguments, in order and with the proper data types. The Label and Names can be whatever you want; they don’t have to match any particular code in your Python script – but the order does matter.

  • Now, click on Execution and in the Script File box, add the Exercise6.py script.

  • Hit OK to add your configured script as a tool in your toolbox.

  • Open your tool from the toolbox, entering the inputs as

    • Class Name: ENV859
    • Distance: 200 Meters
    • A: 100
    • B: 17.6

    Note the format each variable takes in the tool interface and the limitations on what you can enter. (For example, try entering text or a decimal for variable “A” - you can’t! This is all dictated by the variable data type you specified in the Parameters section of setting up your tool.

  • Run the tool and note the messages in the status window (click View Details).

  • Back in your VSCode, edit the Exercise6.py file so that the last two “AddMessage()” commands are now “AddWarning()” and “AddError()”, respectively.

  • Save the code, and back in ArcGIS Pro, run your tool again with the same inputs.

    Note how the addMessage , addWarning and addError messages were presented in the display. Also note the indication that your script has “failed to run” even through it ran as expected. This is because the addError line was run, which always causes the status window to indicate failure.


7. Describing geospatial data objects with the describe object

When writing a script for a certain purpose, you may run into a case where you need to know some property of a given geospatial dataset. What is its extent? Is it a point or a polygon feature class? Is it a floating point or integer raster? How many bands does the raster have? What is its spatial reference?

All these questions are answered by creating a describe object for a given dataset, which is simply a Python dictionary listing all the dataset’s properties. Here is an example of how to create a describe object, here assigned the variable name dsc for the climate.shp shapefile in the V:\ArcPyDemo1data\SanDiego workspace.

More information on describing data using the describe object:
http://pro.arcgis.com/en/pro-app/arcpy/get-started/describing-data.htm

» Exercise 7: Describing data with the describe object

# Scripting with ArcPy: Exercise 7 - Describing data
#
# This script demonstrates how a "describe" object is created and used. 
 
import arcpy
fc = "V:\\ArcPyDemo1\\data\\SanDiego\\climate.shp"
dsc = arcpy.da.Describe(fc)

Once created, the describe object allows us to extract properties of the climate.shp file. These two lines print out the name of the Object ID field (i.e., each row’s internal ID) and whether the climate feature class has a spatial index associated with it.

print (dsc['OIDFieldName'])
print (dsc['hasSpatialIndex'])

To show how this might be use programmatically, we can add these two lines which build a spatial index for the climate.shp feature class, but only if it doesn’t have one already:

if not dsc['hasSpatialIndex']:
    arcpy.AddSpatialIndex_management(fc)

The describe object is a bit tricky in that it is dynamic: its properties vary on what type of dataset is being described. For example, a raster dataset has a “cell size” property, but a feature class would not. However, as the describe object is structured as a Python dictionary, we can easily reveal all the properties associated just by looking at the object’s keys:

print(dsc.keys())

→ Try creating a describe object for the raster “DEM30.img” found in the Data folder. What properties are associated with this dataset?


8. Data access using cursors

Reading and writing data in a feature attribute table is much like reading and writing lines of text in a text file using Python’s “file object”, though there are a few key differences. In ArcPy, instead of a file object, we link to a feature class, table, or raster attribute table using a cursor. These cursors come in three flavors:

Search Cursors are used simply to retrieve individual features;

Update Cursors are used to retrieve and edit properties of individual features;

Insert Cursors are used to append new records to a table and set their values.

More information on cursors is found here:
http://pro.arcgis.com/en/pro-app/arcpy/get-started/data-access-using-cursors.htm

When we create these cursors, we have to specify the data table and the fields in the table we want to retrieve. Optionally we can provide a “where clause” to filter the rows returned. The constraint with cursors, as is the case with file objects, is that data can only be access sequentially: if you want to edit the 100th record, you have to move past the first 99 to get to it. However, supplying the where clause can often be used to narrow in on just the rows you want.

Once a cursor is created, we can use a loop (for or while) to iterate through existing records in search and update cursors. Each iteration returns a “row” object, which is a list of field values created in the same order we specified when creating the cursor.

Here, we’ll only examine the Search Cursor. The ArcGIS Pro online help provides good explanations and examples how the other cursors work. Below is a script example that creates a search cursor for the climate.shp feature class in the SanDiego workspace. (Note that this feature class has two attributes: Climate_ID and Zone which are referred to in the script below.)

» Exercise 8: Working with Cursors

  • Copy and paste the following into a new Python script (or open Exercise8.py)

    #Scripting with ArcPy: Exercise 8 - Using cursors and the geometry object
    #
    # This script demonstrates how cursors are used to
    #  process records in an attribute table, on at a time
      
    import arcpy
      
    # Specify the feature class, raster, or table to examine
    fc = "V:\\ArcPyDemo1\\data\\859_data\\SanDiego\\climate.shp"
      
    # Create a cursor (this is like opening a file)
    rows = arcpy.da.SearchCursor(fc,['Climate_ID','Zone'])
      
    # Use a For loop to iterate through all rows
    for row in rows:
        climateVal = row[0]
        zoneVal = row[1]
        # Print the values for the fields we specified
        print('The climate is {} and the zone is {}'.format(climateVal,zoneVal))
    
  • Run the code and you should see “climate_ID” and “Zone” values returned for the two records in the climate dataset. The result should appear as below::

    The climate is 2 and the zone is Includes climate zones 1 and 2
    The climate is 3 and the zone is Climate zone 3
    
  • Alter the code so that instead of extracting and printing the “climate_ID” and “Zone” fields, you extract and print the FID and Shape@area fields and values. The result should be the following.

    The FID is 0 and the area is 5675863522 m2
    The FID is 1 and the area is 1550644940 m2
    

9. The Geometry object: Reading and Editing feature geometries

In the above example, we modified our code to extract Shape@area to return the area of our feature. The bit after the @ is termed the “token” and there are other “tokens” besides “area” that we can used to pull values from geometry fields (link). However, had we omitted the tokens and just specified Shape@, the value returned would be an ArcPy geometry - in this case a polygon object since our dataset is a polygon feature class.

This is useful because, once we have a specific geometry object, we can do various things to it programmatically…

More on working with geometries is available here:
http://pro.arcgis.com/en/pro-app/arcpy/get-started/reading-geometries.htm

» Exercise 9: Reading Geometries from a Feature Class

  • Alter the code created in Exercise 8 so that Shape@ is included when creating the cursor, and also, in the loop, a variable called feature is assigned to the value in the Shape@ field.

  • Copy and paste the following code to get the polygon object from the first feature in the “Climate.shp” dataset.

    import arcpy
    fc = "V:\\ArcPyDemo1\\data\\859_data\\SanDiego\\climate.shp"
    rows = arcpy.da.SearchCursor(fc,['Shape@'])
    row = rows.next()
    feature = row[0]
    
  • What is the data type of the feature variable?

  • Use the dir() function to list what we can do with this object.

    • What is the length of this feature?
    • Extract the spatial reference of this feature to a new variable; what is the type of that variable?
  • Create a new ArcPy Point object located at X = 6283500, Y=1896150:

    myPoint = arcpy.Point(X=6283500,Y=1896150)
    
  • An now see if that point falls within the feature

    print(feature.contains(myPoint))
    

Point objects vs. Point/Multipoint/Polyline/Polygon Geometry objects

In the above example, the coordinates used to define myPoint are assumed to be in the same coordinate system as feature. But what if they weren’t? How would we specify the spatial reference of our point? The answer reveals a subtle difference between two ArcPy classes related to the Geometry class.

An ArcPy Point object is the most fundamental spatial element in ArcPy. It’s defined by its X-Y (and sometimes Z or M) coordinate properties, but the Point object does not have any spatial reference property to define what coordinate system these values are in.

A Point Geometry object, however, does have a spatial reference property. And thus, if we wanted to query whether a point falls within a polygon and the point’s coordinates are in a different coordinate system, we’d have to first construct a Point object and then construct a Point Geometry object from that point and define its coordinate system. (ESRI seems to go out of its way to make projections, etc. difficult!)

» Exercise 9b: Point Geometries

Let’s look at an example: Use the following code to see if out myPoint defined in geographic coordinates falls within the feature.

lat = 32.8661930
lng = -117.1559074
myPointDD = arcpy.Point(lng,lat)
spatialRef = arcpy.SpatialReference(4326)
myPtGeomDD = arcpy.PointGeometry(myPointDD,spatialRef)
print(feature.contains(myPtGeomDD))

Like Point Geometries, other geometry classes (Multipoint, Polylines, and Polygons) are also constructed of Point objects - or just raw coordinate pairs provided arrays (i.e. Python lists). And spatial references can be assigned to these objects at the time they are constructed or later, via their spatialReference property.


♦ Summary

Clearly, ArcPy does much, much more than what we covered here, but this should at least serve to orient you on how ArcPy is structured and were to go. Really, the key takeaways are:

  • Geoprocessing tools can be run in Python; you just need to know the correct syntax.
  • ArcPy contains other functions besides geoprocessing tools, and they also just require using the correct syntax.
  • Many inputs, including feature and raster datasets, are referenced by file paths stored as string variables. But some, more complex inputs are other Python programming objects, or classes, of which ArcPy includes many.
  • Setting ArcPy environment values via arcpy.env are a useful means to set default values for workspaces, extents, etc for your ArcPy processes.
  • Python scripts can be added to an ArcGIS Pro tool box, with the arguments between ArcGIS and the Python scripts passed as string objects via the sys.argv[] or arcpy.GetParameterAsText() techniques. Also, status and other messages generated in the Python script can be sent back to the ArcGIS Pro application as the tool is running.
  • ArcPy also allows us to programmatically access and describe ArcGIS datasets through cursors - as well as extract, create and edit spatial features via geometry objects.

From here, it’s mostly a matter of getting more experience under your belt to become proficient in using ArcGIS in Python.