Python Debugging: How to Improve Your Skills

When you work with visual programming languages such as Delphi, C#, and VB.NET, you’re a bit spoiled for debugging options as they’re nicely integrated into their IDEs. But when it comes to Python, debugging “out of the box” is a little bit cruder and primitive; single-step debugging is the main way to debug Python code, and is quite slow and clunky. It’s just easier to use print statements; Python founder Guido van Rossum (reportedly) uses them for 90 percent of his debugging.  

In this article, we’ll examine the single-step debugging, as well as other methods for debugging Python code.

PDB

PDB has one feature that distinguishes it from most debuggers I’ve encountered: You can run it in code (it’s a Python class). It supports single-step and view stack frames, and lets you run Python code or call functions. But compared to a visual debugger, it’s a little clunky.

You can extend PDB; for example, there’s PyBug, which simplifies adding breakpoints to your code. There’s also ipdb and pdb++; PyBug will use ipdb (if installed), which can access the iPython debugger with extra features such as syntax highlighting, tab completion and more. iPython is the popular enhanced Python shell, as well as an architecture for interactive computing.

Pdb++ is a drop-in replacement for PDB that adds more commands, sticky mode and tab completion (use the name pdbpp when installing with pip or easy_install).

These are all nice, but they don’t alter the fact that it’s still line-by-line debugging. So what alternatives are out there?

PySnooper

I came across PySnooper recently, and it’s a way of seeing what’s going on inside a function. You import Pysnooper and add either of these lines:

@pysnooper.snoop()   # output to console/stderr

@pysnooper.snoop(‘~/logs/file.log’)  # log file

Now when you run it, it outputs a load of stuff about that function to stderr or a log file path to capture output. With complex functions, the log file is probably the best way to work (rather than to screen). Pysnooper is on GitHub but you can easily install it with this pip command:

pip install pysnooper


Here’s another example, running an interpolation search to find the index of a value from a list. The search algorithm code came from this page:

import pysnooper

import decimal

@pysnooper.snoop()

def InterpolationSearch(lys, val): 

    low = 0

    high = (len(lys) – 1)

    while low <= high and val >= lys[low] and val <= lys[high]:

        index = low + int(((float(high – low) / ( lys[high] – lys[low])) * ( val – lys[low])))

        if lys[index] == val:

            return index

        if lys[index] < val:

            low = index + 1;

        else:

            high = index – 1;

    return -1

print(InterpolationSearch([1,2,3,4,5,6,7,8], 6))

Running it generates this output:

Starting var:.. lys = [1, 2, 3, 4, 5, 6, …]

Starting var:.. val = 6

16:54:02.755170 call         5 def InterpolationSearch(lys, val): 

16:54:02.755473 line         6     low = 0

New var:……. low = 0

16:54:02.755720 line         7     high = (len(lys) – 1)

New var:……. high = 7

16:54:02.756014 line         8     while low <= high and val >= lys[low] and val <= lys[high]:

16:54:02.756216 line         9         index = low + int(((float(high – low) / ( lys[high] – lys[low])) * ( val – lys[low])))

New var:……. index = 5

16:54:02.756487 line        10         if lys[index] == val:

16:54:02.756697 line        11             return index

16:54:02.756925 return      11             return index

Return value:.. 5

5

Whenever a variable changes, its new value is printed out. If you want to see what’s going on inside your function, this is an excellent way—but be warned, it can generate a very large output.

Visual Studio

This is only for those who work with Windows; you can use the free Visual Studio 2019 Community edition and install the Python development workload. Run the VS Installer from the menu, perform any updates, then select modify and enable that workload.

This gives you the full interactive debugging capability, such as single-click breakpoints and view the call stack; you can even run the program in the Interactive window at the same time you step through it.

There is also the cross-platform Visual Studio code (which you can get for Windows, Linux and Mac); it provides similar debugging features as Visual Studio. Python is possibly over-supported, with over 100 free extensions. If nothing else, install Microsoft’s Python extension (and possibly Don Jayamanne’s Python Extension Pack); just search for Python in the extensions search and it will find them and provide one-click installs.

Make sure you follow the instructions for Microsoft’s Python, as it requires you to install Python (but does most of the work for you).

Conclusion: Python Debugging is Worth Learning

I prefer print statements myself to single line debuggers such as PDB, but visual debuggers are much better (as you can see what’s changed, plus you get the enhanced navigation, etc.) It’s just my opinion, but I haven’t found anything that comes close to Visual Studio for debugging, with Visual Studio Code coming a close second on Windows and tops on Linux and Mac.

My preferred method of debugging Python and other languages when a debugger is absent is to add logging. Python has extensive logging facilities and the documentation is superb.

Another Windows-centric technique is sending output via the Windows API call OutputDebugString. This uses UDP (which is like TCP/IP but connectionless, i.e., it can send without a receiver being present). If you build it into your program, it can run quite happily, sending tons of output into “the ether” (so to speak) with minimal impact on performance. If you want to see the output, load a utility that can capture it, such as the excellent (and free) DebugView from SysInternals (now owned by Microsoft) on Windows. On Linux, the tcpdump utility can collect UDP messages.

3 Responses to “Python Debugging: How to Improve Your Skills”

  1. James Igoe

    As to Visual Studio, it might have paid to mention Anaconda. As a hobbyist I’ve used Anaconda for data science for a couple of years, and in a prior role suggested we use it for financial analysts as a way to standardize, and although I think the suggestion was taken up, I had left before it was implemented. Maintaining a consistent desktop for development can go a long way toward reducing confusion and problems.

  2. Douglas Wade Goodall

    The most powerful Python IDE I have used (which is also my favorite) is Wing IDE Pro.

    Wing IDE Pro has one feature that closes the deal for me in a second. It is called remote debugging. The debugger creates an SSH tunnel to the remote machine, and allows you to edit code resident there, and debug line by line (graphically) with breakpoints and a watch window for watching variables. It only takes seconds to set up a remote project and it is very reliable. Better than that though is remote debugging of Python CGI scripts running under Apache on a remote host. This is a miracle as far as I am concerned. It is well past the time we should be writing Python CGI scripts without debugging support. I have been doing Python development lately on a Raspberry Pi 3B+ using the remote debugging capability and it has taken all the pain out of the process. As much as I love the Pi, developing on the graphics console is too slow, and developing in an SSH window with “vi” is too retro. I highly recommend you check out the trial version of Wing Pro IDE. In my opinion, it will change your life. I think it is better than PyCharm.