Approaching a Scripting Task
Introduction: The Challenge 🎞️
Your research team just caught wind that you know Python. They have some Argos tracking data - a text file in a marginally human readable format and with a lot of “noise” - and they want you to build a tool that will parse this data into a format whereby a user can enter a date and retrieve the location at which the turtle was found.
With some trepidation, you accept this challenge. It’s a great opportunity for you to apply the concepts we learned in the last section, namely Python syntax, variables & objects, properties & methods, data structures, etc. It’s also a great opportunity to learn how these elements are assembled into a coherent and effective workflow, i.e., writing a Python script. And of course, you’re bound to learn new tools and techniques for coding in Python!
In this first session, we examine how to approach a scripting task. Writing code is somewhat like translating human ideas into computer logic, and we’ll cover some simple techniques will make that process easier, or at least less intimidating.
Learning Objectives
On completion of this lesson, you should be able to:
- Articulate an example task where Python is helpful.
- Develop pseudocode for an example task.
- Dissect your pseudocode into a coding workplan.
Background - The Overall Task
MoveBank.org hosts tracking data for researchers across the world. We’ll be using a 2019 dataset on black-capped petrels (Petrodroma hasitata) tracked from at-sea capture locations offshore Cape Hatteras, North Carolina, to breeding locations in Haiti and the Dominican Republic. Specifically, we’ll be using data collected using Argos satellite tracking systems.
The data are provided in two comma-separated value (CSV) format files:
Satellite tracking of black-capped petrels 2019-argos.csvincludes the tracking data for the petrels.Satellite tracking of black-capped petrels, 2019-reference-data.csvincludes information on the individual petrels tracked.
➡️ Our task is to construct Python code that can filter the tracking data, listing of all the petrels found within a geographic zone (define by its bounding geographic coordinates) and when they were found there. We want to make this code such that it’s easy to modify the bounds of the geographic selection zone, and if possible, write the selected observations out to a file.
Tackling this task requires some understanding of the raw Argos data, both its format and the data reported. We will discuss this in class and later in this lab. However, all the information needed to do this task is are in the files provided; we just need to develop code to extract exactly the data we want and report it.
Resources
- The black-capped petrel dataset, available here
- A page listing the raw code for specific tasks in this exercise is here.
- A link to a finished Git repository for this project is here: https://github.com/ENV859/Argos_Tracking_Project
Step 1. Generating pseudocode
The context of this exercise is as follows: We have obtained data in a standard format, in this case Argos tracking data. We want to write a Python script that can read any Argos tracking data set, extract records found within a specified geometry (bounding box), and report what individuals were found there and when. Reported observations must also meet quality criteria using tags included in the Argos dataset.
Such a script requires many steps, and to take on all the steps at once is a daunting task. Therefore, we first consider the pseudocode of the exercise, that is, the logical steps involved in meeting our objective given our inputs. In generating pseudocode, we don’t have to know the exact Python commands we’ll implement; we just want to identify the key steps. It’s quite possible we’ll revise some of our pseudocode when actually implementing it, but sketching things out is nonetheless a useful first step in writing a script of any complexity.
Generating pseudocode can be more art than science and gets easier the more you understand particular coding environment. (Think about how much better you are at putting together geoprocessing models now than when you first started learning ArcGIS!) In other words, it may not come naturally at first, but with practice you’ll get better.
I’ve listed the pseudocode for this exercise below:
✅Task 1.1: Develop pseudocode for your project
| Step | Description |
|---|---|
| 1 | Define the selection box as a set of coordinates (x-min,y-min; x-max, y-max). |
| 2 | Open the Argos data to access the data within it. |
| 3 | Iterate through each line in the Argos data file. For each line: |
| 3.1 | - Create a list of data items from the line of text; |
| 3.2 | - Assign variables to specific items: timestamp, latitude, longitude, and species tag; |
| 3.3 | - Determine whether the latitude and longitude falls within the selection box; |
| 3.4 | - If so, print information about the record (tag, date) – or possibly write to an output file. |
Step 2. Devising a plan of attack…
The pseudocode we developed in Step 1 is a blueprint, but not a construction plan.
The next step, therefore, is to map out a sequence - going from simple to more involved - of putting the necessary steps together. The idea here is to isolate the tasks involved in the script into discrete steps such that each step can be built, and debugged in sequence. If we do too much at once and it has issues, we’ll spend more time figuring out where it went wrong. However, if we start simple and something goes awry, our debugging task is more confined.
Determining an appropriate sequence for tackling a project is a bit of an art form, one that you improve with more experience and comfort with the coding language. The general idea, however, is start with a very simple task and add complexity. For example, if you are going to iterate through a chunk of sub-code, first get the chunk of sub-code working, and then add the iteration.
A more relevant example for us is the fact that we need to read text in from a file, and for each line of text in this file, split it into a list of values, and then work with each value. That in itself seems like a tall order, but if we start with just a line of text copied from the data file and pasted into our code - by us, not via Python command - we can tackle the simpler task of parsing that string into a list.
With that in mind, the sequence I suggest we take is as follows:
✅ Task 1.2: Develop the initial construction plan for the project.
| Sequence | Task |
|---|---|
| 1. | Create the coding workspace and initialize the coding script file. |
| 2. | Define the geographic selection box. |
| 3. | Parse a single line of tracking data (copied and pasted into code) into individual variables. |
| 4. | Write code to report whether the tracking record falls within the selection box. |
| 5. | Read a single line of tracking data from the Argos data file and process its location. |
| 6. | Read and iterate through all lines of tracking data from the Argos data file and process their locations. |
| Bonus #1 | Allow the user to define the selection box via inputs. |
| Bonus #2 | Write selected records to an output file. |
There are certainly other ways of going about this, but I’m hopeful you’ll at least see and appreciate the utility of breaking a scripting objective down into sub tasks can really help in writing complex scripts. We will also, almost certainly, deviate from this sequence as we encounter unexpected glitches or possibly improved pathways to our goal. Still, it’s always useful to start with a plan, one that goes from simple to complex.
I should also add that we will encounter plenty of tasks that we have not covered so far (e.g. reading data from a file?!?). But this provides a great way to learn Python - by doing!
Shortly we will embark on writing code to get this done. But first, we are going to explore two important technologies that are going to help us in our coding adventures. First is Visual Studio Code (or just “VS Code”), which will replace our Jupyter Notebook’s interface and offers a host of useful coding tools. And then we’ll examine Git and GitHub, which will track the various evolutionary steps in our code, allowing us to bumble through lots of mistakes and easily recover from them.