The Defense Advanced Research Projects Agency—also known as the U.S. military’s mad-scientist wing—has opened up at least some of the open-source code in its collection for downloading and tinkering.
DARPA’s Open Catalog “organizes publically releasable material from DARPA programs, beginning with the XDATA program in the Information Innovation Office (I2O),” according to the agency’s explanatory note. “XDATA is developing an open source software library for big data. DARPA has an open source strategy through XDATA and other I2O programs to help increase the impact of government investments.”
DARPA went on to claim that it was interested in building new groups and communities on the basis of its software and research; if those communities show enough interest, the agency promises to open up more software, publications, data, and even experimental results.
DARPA’s open-source software could prove useful to firms that deal in data analytics and power computing; downloads include Numba, an open-source NumPy-aware optimizing compiler for Python (sponsored by Continuum Analytics), and WINGS, a semantic workflow system designed to help scientists and researchers map out computational experiments. There’s also Shark, a large-scale data warehouse system for Spark, as well as Tachyon, a fault-tolerant distributed file system designed to work at speed across cluster frameworks.
And those examples are just the tip of the proverbial iceberg—the initial open-source dump includes dozens of software platforms, funded by DARPA and designed in collaboration with either companies such as Boeing, universities, or open-source developers.
Despite its reputation as a place where researchers are primarily concerned with building stealth cloaks and figuring out how to graft laser beams onto sharks’ heads, DARPA has actually invested quite a bit in developing cutting-edge software over the past several years. Last October, for example, the agency offered $2 million to the first outside team that could create automated systems that test software for vulnerabilities and self-generate the necessary computer patches.
As data becomes more integral to modern warfare, it stands to reason that DARPA would invest more time and energy in researching analytics toolsets and ways of more efficiently handling so-called “Big Data.” Private firms could very well benefit from some of the fruits of its research.