This article excerpt is from eFinancialCareers.
If you’re a developer specialized in the Kdb database and its associated programming language, Q, the chances are that you work in financial services. After all, the whole Kdb paradigm was engineered by Arthur Whitney, a former Morgan Stanley technologist after he quit banking in 1993. Since then, Kdb has evolved into the de facto system for banks, hedge funds, and high frequency trading houses looking for fast data extraction and analytics technologies. Now, however, finance firms have competition: their army of Kdb talent is wanted elsewhere.
“Other areas are beginning to catch up [with the need to process large amounts of data]” says Paul Bilokon, a Deutsche Bank director and founder and CEO of Thalesians, a think tank for people interested in quantitative finance, economics, computer science and physics. “I suspect Kdb+/Q specialists will be in high demand in other areas – from civil engineering, to bioinformatics, to medicine.”
A research paper released last year by Bloor, the IT consulting company, underscores the coming threat. Bloor predicted that Kdb’s architecture will increasingly be used across all industries using streaming analytics. What started life as a database system used to underpin banks’ electronic trading and analytics platforms now has applications across the internet of things, the retail sector, self-driving cars, smart technology and industrial automation. The more the world runs on real-time analysis of data, the more that today’s Kdb specialists will be wanted outside banks.
This has the potential to create problems. Banks remain highly reliant on their pool of Kdb expertise. J.P. Morgan, for example, is currently looking for a Kdb developer and data scientist to join its electronic market making team, Bank of America wants a Kdb/Q developer to work on its algorithmic trading systems, Morgan Stanley wants a Kdb developer for its metrics team in New York. The list goes on.
Of course, Kdb isn’t the only database option – there are also Hadoop and SQL and Cassandra and MongoDB waiting in the wings. But the way KX, the company that sells Kdb tells it, the alternatives are all inferior. “Kdb+ is particularly good (both in terms of performance and functionality) at processing, manipulating and analysing data (especially numeric data) in real-time, alongside the analysis of historical data,” says KX, adding that, “The Q language is significantly more efficient than other languages that you might use (both procedural and declarative) for analysis purposes.”
You might say that KX would say this, but Bilokon agrees: “In my personal view, nothing has so far approached Kdb+ in usability, flexibility and speed.”
For more on how recruiters are dealing with the demand for Kdb specialists, as well as how banks and e-finance firms are raising the salaries of these tech pros, check out the article on eFinancialCareers.