Writing & Debugging Scripts

Introduction

This session follows the previous sessions of “Approaching a Scripting Project”, “Using VS Code”, and “Intro to Git & GitHub” where we created our initial project workspace. Here, we do the actual coding exercise which reads in standardized Argos tracking data determines which track records fall within a specified geographic box.

We’ve already developed the pseudocode for this exercise and devised an initial workplan. We’ll apply that workplan here, pausing to learn more Python as needed, and perhaps deviating from our plan to overcome unexpected obstacles.

In doing this exercise, we will also explore how Git works to store and maintain versions of our coding project.

Task Sequence

Set up the workspace and initialize the script.
Define the “geographic selection box”.
Parse a single “hard-coded” line of Argos data and parse text into variables.
Evaluate and report whether the Argos record falls within the box.
Read one line of data in from the Argos data file.
- Side quest #1: How to read text files using Python
Read all lines of data and iterate through them using a for loop.
Read all lines of data and iterate through them using a while loop.
- Side Quest #2: How to revert a Git commit.
Tidy and package our code.

Task 1. Set up the workspace and initialize the script

Coding projects begin with creating your project workspace and linking it to a Git/GitHub repository – something we did in the previous sessions. We still have a few remaining data prep tasks to carry out.

First, ensure that you have a Git-enabled workspace on your machine that has the MoveBank petrel data files, located in the data/raw/MoveBank subfolder. You should also have a README.md file in your project folder. This is the exact same workspace we created in the Git/GitHub lab exercise we just completed.
Open your workspace in VSCode.
Create a new Python script document in the project root folder. Call this “ArgosSelectionTool.py”

Open the script in the VS Code editor and add the following “front matter” to your Python script. (Feel free to modify the format and content.) This comment section provides a quick preview of what this script does, useful in case this script gets separated from its workspace for whatever reason:

#-------------------------------------------------------------
# ArgosSelectionTool.py
#
# Description: Reads in an Argos tracking data file and allows
#   the user to identify the tracked sitings found within a 
#   specified bounding box.
#
# Author: John Fay (john.fay@duke.edu)
# Date:   Fall 2026
#--------------------------------------------------------------
  

Save the file.
Finally, stage & commit your changes to your Git repository (message = “Initial commit of Argos Selection Tool script”) then push those changes to your GitHub repository.
Optionally, sync changes – though it’s not necessary to sync every time you make a commit.

✨Link of what the code should look like after Task 1

Task 2. Define the geographic selection box

To determine whether a petrel observation point falls within a certain geographic selection box, we need to define that box. Leaning into our knowledge of GIS, we know we can do that by specifying the lower left (i.e. southwest) and upper right (i.e. northeast) coordinates. And to keep things tidy, we’ll store these coordinates in a dictionary.

Leave a blank line after the front matter we just added to our script.

Add the following code to create a dictionary object with values for the bounding coordinates:

# Create the geographic selection box
the_box = {
    'x_min' : 34.00,
    'y_min' : -76.00,
    'x_max' : 34.50,
    'y_max' : -75.00
}

Run the code to ensure it has no errors. (It won’t produce any output, but in the interactive window you can run the_box['x_max'] to ensure the dictionary was created correctly. )
Stage and commit your changes to Git.

✨Link of what the code should look like after Task 2

Task 3. Parse a single line of tracking data

In our first coding task, we create a string variable called lineData, setting it equal to a line of Argos data copied from the Satellite tracking of black-capped petrels 2019-argos.csv file and pasted into our script. Then we parse this line of data into its components so that we can print out, in a readable format, information about that record.

In case you are wondering why we are starting with a line of data simply copied from the Argos file and pasted into our script, it’s because it simplifies the task of figuring out how to deal with a line once its read in. Once we nail that, then we should be able to read in all the lines from the input file using the code we develop here.

Add the lines of code below to your script, underneath the “front matter” code added in the previous step. (The pasted code should start at line 12 of your script.)

# Copy and paste a line of data as the lineString variable value
lineString = ''
    
# Use the split command to parse the items in lineString into a list object
line_data = lineString
  
# Assign variables to specfic items in the list
event_id = line_data[]   # Argos tracking event ID ("event-id")
timestamp = line_data[]  # Observation date ("timestamp")
lat = line_data[]        # Observation latitude  ("location-lat")
lon = line_data[]        # Observation longitude ("location-lon")
lc  = line_data[]        # Observation location class ("argos:lc")
tag_id = line_data[]     # Tag identifier ("tag-local-identifier")
  
# Print information to the use
print (f"Record {event_id} indicates {tag_id} was seen at {lat}N and {lon}W on {timestamp}")
  

Fill in the missing code:
- Open the Argos data file in a text editor and copy a data line (not the header line) to the clipboard.
- Paste in the line of data between the single quotes on line 13 of your code (lineString = '' ).
  👉Note that the data itself contains double quotes, so it’s important to enclose our text in single quotes.
- Add appropriate code to split the lineString in subsequent Python statement (lineData = ...)
- Find an insert the correct index values for the variable assignment statements (event_id = ..., etc.).
  👉 It may be helpful to open the CSV file in Excel…
Run the code to check for errors or odd values.
- If you copied the second line from the CSV file (first line of data), the output should be:
```
Record 10154641232 indicates "HA09" was seen at -75.49356999999998N and 34.86216W on 2019-05-14 13:37:52.000
```
- You may want to inspect the value of the variables after running individual or selected lines of code with <shift+enter>.
Stage and commit your changes to Git.

✨Link of what the code should look like after Task 3

Task 4. Evaluate and report if the track point falls within the box

We’ve extracted the coordinates from the pasted line of Argos data. Now we want to see if it falls within the box. GIS software is not required here; we just want to see if the track’s latitude falls between the bounding box’s latitude and the same with its longitude. If so, we’ll report it.

Remove the existing print() statement and its comment line (lines 34 and 35).

Create two Boolean variables to evaluate the latitude and longitude conditions:

#Evaluate latitude and longitude conditions
lat_condition = the_box['y_min'] < lat < the_box['y_max']
lon_condition = the_box['x_min'] < lon < the_box['x_max']

Run the code. You’ll get an error! Why? Because the lat and lon variables are strings, not numbers.

Revise your code so that lat and lon are read in as floating point objects (lines 29 & 30)

lat = float(line_data[3])        # Observation latitude  ("location-lat")
lon = float(line_data[4])        # Observation longitude ("location-lon")

Rerun the code and inspect the values of lat_condition and lon_condition.

Add code to print whether the point falls within the box or not:

#Report the status of the points
if lat_condition & lon_condition:
    print(f'Record {event_id}: {tag_id} was IN the box at {timestamp}')
else:
    print(f'Record {event_id}: {tag_id} was NOT IN the box at {timestamp}')

Test the code. If it looks good, stage and commit to Git.

✨Link of what the code should look like after Task 4

Task 5: Read the data directly from the Argos data file

We now have a simple script that can process one line of the petrel tracking data. But what we really want to do is process all the lines of data, and we are not about to copy and paste each line individually. We therefore need to figure out a way to get Python to read data from the file into our script. That’s not something we’ve covered yet, so we’ll need to learn something new!

🛠️Side Quest 1: Reading text files in Python🛠️

If you were on your own, you might start a web search or using AI to ask: “How do I read a text file into Python?”. And this would likely return with a code sample like this:

with open("file.txt", "r") as file:
    text = file.read()

print(text)

This is close to what you want, but there’s quite a bit going on here. Let’s dissect the code and learn about two things: the “with” keyword and the Python “file object”, starting with the latter.

Exploring Python’s file object

Python’s built-in open() function returns a file object, which has a read() method for reading the content of the file. Let’s try it with our file.

Create a new Python script file in the project root folder, naming it whatever you’d like.

Add the line below to create a file object named f:

#Set a variable to the CSV filename
the_filename = 'data/raw/MoveBank/Satellite tracking of black-capped petrels 2019-argos.csv'
#Create a file object pointing to file name
f = open(the_filename,'r')

Run the line in the interactive window (shift+enter).
In the interactive window’s terminal type help(fileObj) to see the properties and methods related to this file object.
- In particular, note the read() , readline(), readlines() and close() methods.
In the interactive terminal type f.readline() and run it.
Re-run the same command. (🔥Tip: Use the ↑ arrow to scroll to the last command you typed.)

At this point, perhaps you see that the file object’s readline() command reads a line of text from our file and moves to the next line, awaiting another readline command.

Let’s go back to our script. Add the following lines of code:

#Create a list of all the lines in the file via the file object
line_list = f.readlines()
#Close the file
f.close()
#Print the 11th item in the line list
print(line_list[10])

Run all lines of your code. It should print the 11th line of the Argos tracking file by reading all lines of our file into a list object called line_list.
Also note the close() statement. This is important as it releases Python’s hold on the file, e.g. so that other programs can access it exclusively.

👉Using the file object to write and update files

In creating the file object, we added the argument r, specifying that we were opening the text file only to read it.

The w argument would open the file for writing. ⚠️Be careful with this, however, as it will wipe the contents of any existing file with the file name you provided.⚠️

The a argument allows you to append new text to your file.

This site offers a great tutorial on this: https://www.w3schools.com/python/python_file_write.asp I’m also happy to review these techniques with you in class.

The `with` keyword

Python’s with keyword is useful when you only need an object temporarily in your script. Let’s try it and then explain how it works.

Change the code in your demo script to this:

with open('data/raw/MoveBank/Satellite tracking of black-capped petrels 2019-argos.csv','r') as f:
	line_list = f.readlines()
print(line_list[10])

Run the code to ensure it produced the exact same output.

Notice that we still create the file object, assigning it the variable name f, but f is only referred to in the code chunk indented under the with keyword. Also notice that we omitted the close() statement. This is because once the code chunk associated with the with statement completes, the object created, here f, is destroyed. Thus, there’s no need to close the file. And that’s why file objects are often created with the with keyword, even though it’s a bit less intuitive to read.

Task 5: Resumed

Now that we have a handle on Python’s file object, let’s return to our ArgosSelectionTool.py script, setting it to read the data into our script in place of just copying and pasting it in. The code snippet below can serve as a template for what’s next. You can copy and paste this into your script just after the code creating the selection box dictionary (and replacing all existing code after the front matter).

#Create a variable pointing to the data file
file_name = '█'

#Read the contents of the file into a list of lines
with open(█,'r') as f:
    #Read contents of file into a list
	line_list = f.█

#Pretend we read one line of data from the file
lineString = line_list[█]

# Use the split command to parse the items in lineString into a list object
line_data = lineString.split(',')
  
# Assign variables to specfic items in the list
event_id = line_data[0]   # Argos tracking event ID ("event-id")
timestamp = line_data[2]  # Observation date ("timestamp")
lat = float(line_data[3])        # Observation latitude  ("location-lat")
lon = float(line_data[4])        # Observation longitude ("location-lon")
lc  = line_data[14]        # Observation location class ("argos:lc")
tag_id = line_data[-3]     # Tag identifier ("tag-local-identifier")
  
#Evaluate latitude and longitude conditions
lat_condition = the_box['y_min'] < lat < the_box['y_max']
lon_condition = the_box['x_min'] < lon < the_box['x_max']

#Report the status of the points
if lat_condition & lon_condition:
    print(f'Record {event_id}: {tag_id} was IN the box at {timestamp}')
else:
    print(f'Record {event_id}: {tag_id} was NOT IN the box at {timestamp}')

Now, you’ll have to make the following edits (where you see the █ character) so that it runs correctly:

Set the file_name variable to a string indicating the location of the Argos data file. This path will be relative to our script file, so the full relative path will be “data/raw/MoveBank/Satellite tracking of black-capped petrels 2019-argos.csv”.
Next, set the f variable to be a Python “file object” created by opening the file who’s path is stored in the file_name variable in “read-only” mode.
Apply the readlines() function to the file object to read it’s entire contents as a list of lines stored as the line_list variable.
Now, instead of assigning the lineString variable to a string that was copied from the sara.txt file and pasted in your script, assign the lineString variable to the 201st item in the line_list list object.

The remaining lines are the same as in the previous script…
Run the code, and if it runs successfully, commit the changes to your Git repository.

✨Link of what the code should look like after Task 5

Task 6: Process all lines in the Argos file using a `for` loop

Now let’s expand on what we did above and loop through all lines in the file. To do this we’ll replace the line were where we extract one line from our line file (by its index) with a “for” loop that iterates through all [data] lines and processes each just as we did the one.

Change the line lineString = line_list[200] with for lineString in line_list:.

Select all lines below that and indent them (by hitting the tab key when selected in VSCode).

#Loop through all the lines
for lineString in line_list[1:]:
   
    # Use the split command to parse the items in lineString into a list object
    line_data = lineString.split(',')
       
    # Assign variables to specfic items in the list
    event_id = line_data[0]   # Argos tracking event ID ("event-id")
    timestamp = line_data[2]  # Observation date ("timestamp")
    lat = float(line_data[3])        # Observation latitude  ("location-lat")
    lon = float(line_data[4])        # Observation longitude ("location-lon")
    lc  = line_data[14]        # Observation location class ("argos:lc")
    tag_id = line_data[-3]     # Tag identifier ("tag-local-identifier")
       
    #Evaluate latitude and longitude conditions
    lat_condition = the_box['y_min'] < lat < the_box['y_max']
    lon_condition = the_box['x_min'] < lon < the_box['x_max']
   
    #Report the status of the points
    if lat_condition & lon_condition:
        print(f'Record {event_id}: {tag_id} was IN the box at {timestamp}')
    else:
        print(f'Record {event_id}: {tag_id} was NOT IN the box at {timestamp}')

Run the script with these changes. You get an error with the very first line read! Why? Well, that’s the header line isn’t it! We need to skip that. To fix that we can modify the line for loop to skip the first line:
```
for lineString in line_list[1:]:
```
Make that change and re-run the script. You get another error about 28 lines in! Why? Inspect the raw data (e.g. in Excel) and you see not all records have coordinate information.

Recalling that these are Argos data, meaning not all records are valid, we can use the Argos reported location class (lc) to filter which records get processed.
Add code such that if the lc value is not 1, 2, or 3 (i.e. reliable records), we’ll skip processing them:
1. Move the line that extracts the lc value above the ones that read (and convert to float) the lat and lon values. (Do you know why?)
2. Immediately below that, add a conditional statement to skip processing if the lc is not ‘“1”’, ‘“2”’, or ‘“3”’.
```
    lc  = line_data[14]        # Observation location class ("argos:lc")
    if not lc in ('"1"','"2"','"3"'):  continue
```
Optionally, remove (or comment out) the code that reports records that are no in the box.
Test that the code is operating as expected; if so, stage and commit the changes to Git.

✨Link of what the code should look like after Task 6

Task 7: Process all lines in the Argos file using a `while` loop

The for loop works quite well, but for it to work, Python has to store the entire contents of the text file into the computer’s memory. That’s fine in our example, but what if we had an enormous file? A while loop, combined with using the file object’s readline() method (vs readlines()), allows us to process just one line at a time, bypassing the need to load the entire file into memory. Thus, it’s good to know how to do this…

To implement a while loop in this code:

Change Line 26 to lineString = f.readline() (We just want to read one line at a time in our while loop…)
- You may also want to update the comment above the line…
Indent all lines below the with statement. (We’ll need to keep the file object open for all of thes processing.)
Replace the for loop (and comment - lines 20 & 21) to a while loop:
```
#Iterate through lines
while lineString != "":
```
What would happen if we ran the code right now??
At the end of all indented lines (should be around line 53) , insert the following code making sure it’s indented to run within the while loop:
```
      
    # Move to the next line
    lineString = f.readline()
```
Test the code. Nothing appears to happen! This is because your script is in an infinite loop. See if you can debug and fix this by stepping through the code using the debugger.
Whether or no you get the code to run correctly stage and commit the changes to Git and push all changes to GitHub.

✨Link of what the code should look like after Task 7

🛠️Side Quest 2: Reverting a Git commit🛠️

Say you actually did want to use a for loop in your analysis, not a while loop. This is where Git is handy. Unfortunately, the Git interface in VSCode is not as friendly to do this, but we can always use Git commands to manage our repository. In this case, we’ll use the git revert command to undo a specific commit in our repository. To do this, however, we’ll need to get the ID of the commit we want to revert.

Push any local commits up to GitHub
- Ensure that your local git repository is clean, i.e., no uncommitted files.
Find the ID of the commit to revert – from GitHub.com
- Navigate to your repository on GitHub.com
- Below the green Code button you’ll see a listing of the number of commits you’ve made in your repository. Click that to get more info on each commit.
- Each commit is listed with its message and on the right side an ID. Just to left of each ID (called a “SHA”) is an icon you can click to copy the SHA to the clipboard.
- Copy the SHA associated with the While loop commit to your clipboard.
[Alternate] Find the ID of the commit to revert – from VSCode
- In VSCode, open the Explorer pane by clicking the top icon in the Activity Bar
- At the bottom of the pane is an entry called GRAPH. Click that while your script is active.
- This list all the saves and commits to your file. Right click the commit associated with the While loop commit and select “Copy Commit ID”.
Undo the commit
- Open a new terminal in VSCode.
- Type git status to ensure Git is receiving commands.
- If all is good, type git revert followed by the SHA you obtained above, and then ending with --no-edit
```
git revert a7adcd81d6a737d63f6d8a4df2a7787145a2b622 --no-edit
```
- Have a look at your code: it should have reverted back to the for loop.
Push your “reversion” to GitHub
- Open the Source Control pane in VSCode and push (Sync Changes)
- Check your GitHub site to see that the latest version has the for loop, not the while loop.
👉 The git revert doesn’t actually remove the commit, but rather it creates a new commit that undoes the changes in the commit we specified. This is in line with the notion that Git really doesn’t like to lose versions…