Using Python for High-Performance Frameworks


What About Non-Web?

Not everybody does web development. Suppose you need to do some serious number-crunching and your algorithms function in parallel. Why not make use of all those cores? While this is an exciting area to be working in, utilizing Python in this context presents some difficulties. If you’re interested in exploring multicore programming, what I recommend is an approach likely to make people unhappy:

First, learn the basics of parallel programming using a language such as C++, combined with tools that support multicore programming (such as OpenMP).

Once you’re familiar with the ins and outs of multicore programming, you can start work in Python, recognizing some of the difficulties involved and possible ways to overcome them. For a fantastic discussion on those challenges and workarounds, check out Nick Coglan’s article on multicore python programming.

When it comes to high-performance computing within the context of Python, a technology called SIMD Single Instruction, Multiple Data) is having a big impact; it refers to a processor’s ability to perform a computation on an array of numbers with a single assembly-language instruction. SIMD, which is not new by any means (it was used in the 1970s), is also known as vectorization, because you’re operating on vectors of data. If this is a topic you’re interested in, then check out this blog about PyPy, as well as this blog about IPython and NumPy.


At present, much high-performance programming is being done in C++ because of the closeness to the processor and the ability to spawn threads across cores. But higher-level languages are catching up. Python now has asynchronous programming, and there are ongoing efforts to help you write multi-core and vectorized programming in various dialects and implementations of Python. Explore a bit, and you might be pleasantly surprised by what Python can offer.

One Response to “Using Python for High-Performance Frameworks”

  1. James Chau

    35 years of asynchronous process programming (called pseudo-conversational programming in the 1980’s) and now the programming fields and academia are now talking it up again. 35 years of IBM CICS online pseudo-conversational programming, smooth as silk.