Python Developer Job Market Having a ‘Hockey Stick’ Moment

Perennially popular programming language Python may be a fan favorite, but can you make a living as a Python developer? More to the point, how is the job market holding up for one of the world’s most popular languages? As it turns out, you can’t go wrong with Python – and recently, things feel very right.

Digging into numbers within the Dice database, we isolated “Python developer” jobs. We also ‘cleaned’ the data so other jobs referencing the language by name were removed from this study. (We did this because some jobs list Python as a preferred skill, but those positions don’t focus on writing apps or services in the language full-time; it simply felt disingenuous to include all jobs referencing the language.)

You may be unsurprised to know the Python job market is very healthy. Not only is ‘Python Developer’ one of the more popular jobs in our database, it’s also one of the steadiest, historically speaking. While many job titles and programming languages (such as Kotlin) are subject to industry whims and the decisions of large tech companies, Python weathers storms with ease, apparently unfazed by industry trends.

And the past three quarters have seen a marked spike for Python jobs. As you see in the chart below, the ‘Python Developer’ job title has returned rock solid numbers from the beginning of 2016 (Q1 2016, specifically). From Q3 2017 to the second quarter of 2018, Python Developer jobs have almost doubled. (Remember, this data is normalized, so our findings here are specific to Python Developer roles. The actual counts are far higher if we include all jobs utilizing Python.) We don’t yet have data for the third quarter of 2018, though our earliest dive into the dataset doesn’t suggest we’re going to see a precipitous drop-off. Things are still looking up!

The language seems to be one that’s found a use-case in just about every corner in tech, though nothing specific seems to be responsible for this recent spike. That being said, there’s one increasingly popular discipline providing a robust new avenue for Python devs everywhere: machine learning.

The R programming language is specifically geared toward statistical computing, which makes it a prime candidate for machine learning jobs – but Python may be usurping it at just about every level. Speaking with Dice, Enriko Aryanto, CTO of QuanticMind, said: “R has issues with scalability. It’s a single-threaded language that runs in RAM, so it’s memory-constrained, while Python has full support for multi-threading and doesn’t have memory issues.”

Learning Python isn’t difficult; it’s easily understood and fairly verbose, and writing simple applications or services is attainable for beginners. The language is taught early in schools, and is one of the most widely used in the open source community. All told, the Python programming language is in a great position. It’s already one of the most popular languages or skills around, and analysis shows its job market is steady.

13 Responses to “Python Developer Job Market Having a ‘Hockey Stick’ Moment”

  1. Jeff Jones

    Let’s look at this from a business (i.e. a revenue) perspective.

    I am a project manager, responsible for a project’s budget, deliverables, understanding and meeting legitimate business concerns (like time to market). I have a new project and I need to select a technology to use for development, which includes selecting a language.

    Why would I choose Python? Python is not typesafe, it is interpreted (which makes it slower and not as scalable). I can find a lot more .NET/C# programmers or Java programmers than I can Python. Visual Studio gives my developers a shortened design, coding, and testing portion of the SDLC than Python.

    In the end, going the Python route provides no benefits but costs me more in time and money.

    Aside from the nebulous “everybody is doing it” excuse, why would I choose Python?

        • Jeff Jones

          And how much initial and operating cost do they pour into the system that runs it because of a poor choice when they started? A real engineer looks at life cycle cost as part of making a decision.

      • Python can be compiled or interpreted, however the meaning of those words is frequently confused. Neither .NET nor Java are technically compiled. They are both transpiled to an interim language, CLR and Java Bytecode respectively. These are then “interpreted” and just-in-time compiled to run on virtual machines, the .NET run time or the JVM.

        Python can be typed, see the “typing” library and the Mypy static analysis tool. It can also directly use fast foreign code libraries like C or Fortran or can be both strongly typed and faster by using Cython.

        Python is as scalable as you make it, as are most languages. Testing, developing and designing are all mature processes in Python.

        R can be multithreaded, and there is active work to port existing libraries to parallel execution versions.

        As a project manager, you should choose a language that is easy to hire for, easy to develop in and easy to maintain, and is not a fad. .NET, Java, Python all qualify.

    • Armando Alejandro

      You can install Python as a language option in Visual Studio now, so that gives python developers the same development environment as other .Net languages.. I think it’s use is more with start ups, but even the large companies like Facebook are using it now for certain types of development. But I think if you’re building a website .Net MVC is still the way to go.

      • Jeff Jones

        Same basic IDE, but not all the functionality and compile capability a VS user gets with C# or VB. Nor do VS IDE Python users get all the capabilities of a more versatile language like C# and VB.

    • code_dredd

      @Jeff Jones

      > I am a project manager, responsible for a project’s budget, deliverables, understanding and meeting legitimate business concerns (like time to market).

      That’s all understandable.

      > I have a new project and I need to select a technology to use for development, which includes selecting a language.

      I think the decision of which technology to use should be left to the more experienced engineers, because they’re the ones that are actually going to be doing the work. If, as a project manager, you’re telling your engineers what language and/or technology they should be using, quite frankly, you’re overstepping your boundaries and might end up creating problems by choosing technologies that your more experienced engineers won’t be familiar with and won’t be able to provide technical guidance to the more junior developers, etc.

      In general terms, as a manager, you should set the mission goal, and then get out of the way. This is not to say that you should go into hiding; ask questions and get your engineers to justify things so that everyone can see whether you’re standing on solid ground or might be heading towards a wreck.

      > Why would I choose Python? Python is not typesafe

      So? Why is that a problem and why is that _your_ problem to be concerned about as a project manager and not a software engineer?

      Assuming your team has applied proper engineering practice and used automated testing to validate their code (e.g. unittest package), this is a non-issue.

      > it is interpreted (which makes it slower and not as scalable).

      Yes, it is interpreted, and yes, interpreting the code makes it slower than compiling to native code, but this is a non-sequitur. The first two facts do not imply your conclusion that it is “not as scalable”. Additionally, if your main rationale for not using Python is that it is “slower” than some other language, then you’d have to argue the same point against someone trying to use Java or C# because those languages are actually interpreted too (by the JVM and CLR respectivelty): “Why use Java/C# when it’s slower than C?” “Why use C when it’s slower than x86 assembly?”.

      As a project manager, you’re presumably familiar with the concept of tradeoffs.

      Python is quite scalable and is even used in scientific applications and HPC. In addition, a lot of scalabity today goes horizontally rather than vertically, meaning that the scalabity comes from exploiting parallelism across many CPUs/Cores rather than running as fast as possible in a single one. (e.g. see the multiprocessing module)

      Also, if you look at companies that have had to solve scalability problems (e.g. Amazon, Facebook, etc), you’ll see that they’ve used a lot of Python to tackle those problems.

      Lastly, talking about speed without profiling to see the differences borders in nonsense. Slow programs can also be implemented in C, simply by choosing algorithms that perform poorly for the given task when there could’ve been a better one (e.g. “better” in the sense of a lower asymptotic upper bound for a given input size).

      > I can find a lot more .NET/C# programmers or Java programmers than I can Python.

      If that’s important to you, then why the question of whether you should or shouldn’t use python? In this case, it really has nothing to do with the language and everything to do with the “talent pool”.

      > Visual Studio gives my developers a shortened design, coding, and testing portion of the SDLC than Python.

      It’s been several years since I’ve used VS, but I think the idea of comparing an IDE with a language is flawed; you’re _not_ comparing apples to apples. There’re good IDEs for Python as well (some of my co-workers swear by PyCharm, even though I’ve never used it myself).

      If you want your comparison to be valid, then compare languages with languages.

      > In the end, going the Python route provides no benefits but costs me more in time and money.

      What will really provide no befenits and cost you more time and money is trying to micro-manage the technical aspects of the project instead of trusting your more experienced engineers to make good decisions in their areas of expertise.

      > Aside from the nebulous “everybody is doing it” excuse, why would I choose Python?

      You shouldn’t. You should let your more experienced engineers make that decision. It’s them that’ll be doing the actual work, not you. On the flip side, if you need additional convincing, you free yourself from possibly setting them up for failure due to a poor decision on your part. Everyone wins.

      • Jeff Jones

        Thanks for the reply. You make some good points. I failed to mention that I am also a seasoned software engineer with other engineering experience outside software.

        > the decision of which technology to use should be left to the more experienced engineers
        I am about as experienced as they come. I always listen to the other engineers on the team, but in almost all cases, I have more experience and knowledge. If I see a solid, definable reason to choose something different than their consensus, then I explain it (teachable moments) and we do it.

        >Yes, it is interpreted, and yes, interpreting the code makes it slower than compiling to native code, but this is a non-sequitur. The first two facts do not imply your conclusion that it is “not as scalable”.

        Actually it does. One of the benefits of being an engineer as well as a project manager. As concurrency grows (i.e. more users, more complexity, etc.), interpreted languages require more hardware to perform even close to compiled languages. Changing hardware or scaling up existing hardware takes time – not enough time to be responsive to growth (mostly due to the budget approval and purchase process). Using a compiled language decreases the impact of increased usage, and decrease overall hardware costs.

        >… if your main rationale for not using Python is that it is “slower” than some other language, then you’d have to argue the same point against someone trying to use Java or C# because those languages are actually interpreted too (by the JVM and CLR respectively):
        No, that is incorrect. C# is compiled when it executes. Java has the option of being compiled when it executes, though it usually is not.

        >“Why use Java/C# when it’s slower than C?” “Why use C when it’s slower than x86 assembly?”.
        Because the gain going from C# (which is overall faster than Java in production and requires less developer time) to C or C to assembly is negligible. The added lifecycle cost for coding only in C is much higher than using C# or Java and beefing up the hardware a bit. I don’t know if you have studied Value Engineering, but if not, you should. It brings a great perspective to technology decisions. Value Engineering was most famously a critical part of the US space race and the US Naval Nuclear Power Program successes. If you know anything about Clarence “Kelly” Johnson and Lockheed Skunkworks, it was critical to their success (https://www.lockheedmartin.com/en-us/who-we-are/business-areas/aeronautics/skunkworks/kelly-14-rules.html).

        >Python is quite scalable and is even used in scientific applications and HPC. In addition, a lot of scalabity today goes horizontally rather than vertically, meaning that the scalabity comes from exploiting parallelism across many CPUs/Cores rather than running as fast as possible in a single one. (e.g. see the multiprocessing module)

        Not really. First, no one thinks of scalability vertically any more. Second, if on a given hardware platform, Python can handle X concurrent users, C# or Java can handle X * 1.y users. Better value, and lower life cycle cost.

        >Also, if you look at companies that have had to solve scalability problems (e.g. Amazon, Facebook, etc), you’ll see that they’ve used a lot of Python to tackle those problems.

        Never underestimate the negative effects of the religious technology wars. Python was a good choice for those who need the statistical capabilities, but have a religious ethos of “Anything but Microsoft”. A good manager, as well as a good engineer, steers clear of those religious arguments.

        > What will really provide no befenits and cost you more time and money is trying to micro-manage the technical aspects of the project instead of trusting your more experienced engineers to make good decisions in their areas of expertise.

        True for project managers that are not experienced engineers. My decades of experience have reinforced the idea that technology projects should be managed by engineers with solid business and management abilities

        So I stand by what I wrote, being both an experienced engineer and project manager..

        • code_dredd

          @Jeff Jones

          > I am about as experienced as they come. I always listen to the other engineers on the team, but in almost all cases, I have more experience and knowledge. If I see a solid, definable reason to choose something different than their consensus, then I explain it (teachable moments) and we do it.

          That’s fine, but based on your description, your role is no longer that of an engineer and you’re no longer doing the engineering work; you’re doing managerial work now. Still, while what you’re saying might be true and correct here (hard to tell without concrete examples), I also have my share of experiences where engineers who think are more experienced and, thus, think of themselves as more knowledgeable, really aren’t. More experience does not automatically mean “more knowledgeable”; it’s often the opposite as people get complacent in their comfort zone and don’t even realize how far behind they’re really lagging.

          I’m *not* saying this is your case; only that an appeal to authority is not a good argument to make and I’m not the kind of person to fall for those. I could also mention how I’m also an experienced computer scientist doing software engineering work for over a decade including in Fortune 500 companies, but that’s not really relevant to the point at hand.

          > Actually it does. One of the benefits of being an engineer as well as a project manager.

          Another appeal to authority without any reasoning to justify why “it does” other than, well, “it does”. Also, given the topic, the only relevant part of your background here is the engineering one, not the managerial side.

          > As concurrency grows (i.e. more users, more complexity, etc.), interpreted languages require more hardware to perform even close to compiled languages. Changing hardware or scaling up existing hardware takes time – not enough time to be responsive to growth (mostly due to the budget approval and purchase process).

          You’re correct, but are failing to address what I actually said and appear to have missed the point due to by the fact that you’re focusing on constant factors. My point regarding scalability was: asymptotic growth matters more.

          Throwing hardware at the problem is, in many cases, a sign of the fact that the reality of asymptotic growth is rearing its head and that people who’re supposed to know better don’t seem to realize what the real problem is. That’s why I made a point of prioritizing data structures and algorithms, which are the most influential factors (not only factors) in scalability.

          As an example, there was a project about sequencing by hybridization (SBH) where working on a string with 4,096 characters took > 2 days when using a binary tree, ran out of memory when using a suffix tree, but completed in ~45 seconds when using a compressed tree… all in the same hardware and using the same language. (The Algorithm Design Manual 2nd Ed, Skiena. S, p.98)

          > Using a compiled language decreases the impact of increased usage, and decrease overall hardware costs.

          Perhaps, but only by a comparatively negligible *constant* factor. The referenced case above should be enough to make the point, especially since you claim to be an experienced software engineer, and should be more than familiar with these basics.

          In short, if you want “scalable”, but then turn around and focus on whether the language is or isn’t interpreted, then you’re focusing on the wrong thing first. You should leave that item for last in your list, after you’ve already exhausted everything else, and even in that case, you only get a one-time “boost”, and this does not and will never “scale” with the size of your inputs.

          > No, that is incorrect. C# is compiled when it executes. Java has the option of being compiled when it executes, though it usually is not.

          You’re either playing semantics or are not as technically knowledgeable here, which is important, since our discussion is about languages.

          C# is not compiled to native code; it is “compiled” to an intermediate language called MSIL which is itself interpreted at run-time, just like Java is “compiled” to intermediate byte-code that is interpreted at run-time. By your own line of reasoning here, you’d also have to argue that Python itself is a “compiled” language. Whether Java gives you additional options to tweak some things is irrelevant to the point.

          > Because the gain going from C# (which is overall faster than Java in production and requires less developer time) to C or C to assembly is negligible.

          When I said “Why use Java/C# when it’s slower than C?” “Why use C when it’s slower than x86 assembly?” I was not asking questions; I was trying to illustrate the issue with your line of reasoning, which is that you were using “that language is slow” as your justification for choosing a different one, which is fine, but then you seemed to *arbitrarily* stop applying the same analysis to your new choice, even though it also applies there.

          > The added lifecycle [sic] cost for coding only in C is much higher than using C# or Java and beefing up the hardware a bit.

          I’m inclined to agree. However, if you carefully read what I said, you’ll notice that I was not stating that you should’ve been using C or assembly. I think you missed the point I had made w.r.t. your line of reasoning.

          > I don’t know if you have studied Value Engineering, but if not, you should.

          That was an interesting read. I have not formally studied VE, but I’m familiar with the intended goal of the process.

          > Not really. First, no one thinks of scalability vertically any more.

          So, you deny what I said, and then state exactly what I had said? I specifically said that “scalabity today goes horizontally rather than vertically”, so how is your attempted refutation here any different from what you were, presumably, trying to refute?…

          > Second, if on a given hardware platform, Python can handle X concurrent users, C# or Java can handle X * 1.y users. Better value, and lower life cycle cost.

          I think you’re deeply confused here.

          You are conflating the *language* with the *run-time*. Python (the language) has different run-time implementations, such as CPython (the one you seem to be conflating with the language), Jython, IronPython, etc.

          CPython has no JIT like JVM and .NET do, for example, but that’s a run-time issue, not a language issue. Jython is interpreted by the JVM and has the same run-time profile as any other Java program. IronPython relies on the same .NET libraries and IL as C# does, so the performance profile should be basically the same as any other C# program. Python can also be translated to native code with other tools, getting you a performance profile similar to that of C++.

          In short, your reply and justifications are based on a deep misunderstanding.

          > Never underestimate the negative effects of the religious technology wars.

          Who’s being “religious” about anything? It makes no difference to me whether you use or refuse to use Python. I’m simply noting that the reasoning you’re using is flawed and that you have several misunderstandings without realizing it. We really could’ve been talking about any other language.

          > Python was a good choice for those who need the statistical capabilities, but have a religious ethos of “Anything but Microsoft”. A good manager, as well as a good engineer, steers clear of those religious arguments.

          Sure, but you’re the only one making references to religious wars here… this is a red-herring.

          > True for project managers that are not experienced engineers. My decades of experience have reinforced the idea that technology projects should be managed by engineers with solid business and management abilities

          No offense, but based on the technical inaccuracies and misunderstandings I saw in some of your comments, maybe you should be more aware of the potential to over-estimate your own abilities[1], and/or maybe even just lagging behind technical changes due to a change of primary focus (i.e. from engineering to management), and the impact that may have.

          [1] If you haven’t read about the Dunning-Krueger effect, I suggest you take a brief look. (https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect)

          > So I stand by what I wrote, being both an experienced engineer and project manager

          Power to you. It’s your project and your team, not mine. 🙂

    • Penguin Monkey

      There have been number of studies and only 15% of errors can be checked by conventional type systems even if they have the null checks that the new type systems have. For example, google the JavaScript to TypeScript bug checks, what percent of bugs are TypeError, AttributeError, or NameError in Python, the converting Erlang/Python to Haskell code studies. Its also possible that the static types of C# and Java cause people to write more boiler plate (code that cannot be factored out) and thus increase the number of errors in the 85%. Try doing the stuff you can do with Python’s __getattr__, methodmissing, or JavaScript’s prototypes in Java. Only the inner loops (10%-20%) of code have to be optimized, and if you look at a profiler some functions are being run 10,000 times more than others so focus on optimizing those. Python can offload many of its inner loops through using sum, product, and, any, groupby, numpy, etc… Dynamic languages can seem fast despite being CPU slow because its easier to load modules and data “lazily” and they give you more indirection by default, for example, JavaScript beat Java in the browser because the JVM startup time which was loading a lot of stuff from the hard drive. To the user “JavaScript” is faster than Java, although to the CPU Java is faster.

  2. Alexander Sterczek

    Although Python can be considered efficient and fast for a vast number of web applications; because it executes using an interpreter instead of a compiler, this causes delays in processing.

    • Penguin Monkey

      Actually python is compiled while its being interpreted. It creates .pyc files which are then run on the python virtual machine, and it does just-in-time optimization but its nowhere near Lua, .net, JVM, JavaScript engines.