Some say, “You get what you pay for.” Other times, you hear, “The best things in life are free.” When it comes to incorporating open source into your software development project, which is correct? Not unlike many situations in business and in life, the answer is actually: “It depends.” Open source is no longer relegated to the discount software vendor that serves cash strapped startups. In enterprise software development these days, open source is not only immensely valuable, but increasingly
crucial to stay competitive in releasing high quality software at regular intervals in a world where technology is changing so fast and every edge matters. Today, rolling your own logging package instead of using something like
log4j is as silly as trying to build your own web server instead of using Apache
httpd was 10 years ago. Still, there are other components like
guava that are less well known, but are currently making a name for themselves as libraries that can take the solution you are building to the next level of sophistication and quality. Just knowing they exist—and knowing where they fit—can help you design and build better software at a lower cost.
Evaluating a Component to Include In Your Project
In addition to conducting a traditional build versus buy analysis, it’s critical to think about the maintenance and support story surrounding an open source package. Some things to consider and questions to ponder:
- What sort of licensing does the component offer? Is it copyleft or more permissive?
- How well written and understandable is the documentation? Is the code sensibly organized and commented? Can you quickly and easily build a binary distribution from source?
- How active is the user community on the project’s own website, and/or independent Q&A sites like StackOverflow?
- Are there regular code submissions by the developer community on SourceForge, Google Code, or GitHub?
- Do committers and contributors respond to forum posts or mailing list inquiries?
- Is there a web accessible bug ticketing system that you can easily browse?
- What sorts of consultants or systems integrators have expertise in using the package? Are they available for hire and are they reputable? How much would a service engagement cost your organization if you really needed it?
- Is there a parent company that has taken stewardship of the project and is actively sponsoring it with developer time or other resources? Examples of this include how IBM contributes developer time toward Eclipse or Google toward GWT.
- Are there companies out there that offer certified enterprise grade, packaged versions of the software? What specific SLAs are available on such packages? At what cost? A great example of this is what RedHat does with Linux, among many examples out there.
- In the event that no other help is available, do you have expertise on your engineering staff to patch the code and rebuild a binary version? Will your developers have the necessary skills and dexterity with languages and toolsets to modify, build, and deploy the patched binary with confidence?
Case Study
I’d like to share my experiences of using open source in general, and one in particular, to shed some light on what it’s like. We use some open source components and tools in building the flagship product at my company. With established processes in place about how we handle open source, our experiences navigating this vast ocean we call “open source” have been positive and we are grateful for the contributions the community has made to advance our field. What I’m about to relate to you is an
atypical experience, but one that may be similar to the situation you may well find yourself in when looking to incorporate open source technology. In the upcoming release of our latest platform, we will be incorporating some additional performance monitoring capabilities that will allow our customers to better gauge the health of one of their deployments on the platform. After some consideration, we decided it would be a good idea to leverage the
Hyperic SIGAR API, which is now part of the
VMWare vFabric Hyperic product line and offered independently under the
Apache License, version 2.0. I spent some time using the API and incorporating the library into our metrics collection code. It turns out that this library has both a Java as well as a native C/C++ component that is invoked via
JNI (Java Native Interface). After a while, we noticed a bug in this code that intermittently crashed the entire JVM. This tight coupling is one of the drawbacks of using JNI technology – a crash in native code means a crash in the entire Java process. Unfortunately, this crash was occurring on only one platform (x64 Windows) and
very intermittently - which made it quite difficult to debug. My first inclination was to do some research online to find out if anyone else in the user community had this problem. I searched the
SIGAR JIRA tickets, to no avail. I then searched the
SIGAR User Forum and found out
someone else was seeing the same crash that we were at the same hexadecimal offset in the DLL. I asked for advice and help on the
SIGAR User Forum and the
SIGAR Developer Forum and even e-mailed the lead developer directly. No solution was in sight. To make matters worse, it seemed as if
active development on SIGAR had stalled (or at least significantly slowed). We reached out to VMWare and they said, while they don’t have an enterprise SLA available for purchase for SIGAR, they could connect us with a third party consultant who might be able to help. At this point, we had 4 options:
- Take our chances with finding and paying for a third party consultant who could help us.
- Build our own SIGAR-like functionality from scratch.
- Look for alternatives to SIGAR.
- Debug and patch SIGAR ourselves.
I spent some time on alternative #3, but quickly discovered that SIGAR was the most mature product in its class. I then decided to spend some time on alternative #4. Like others in the community,
I initially had some trouble building the binary from source by simply following the instructions. However, after some detective work,
I discovered a missing file was the culprit and eventually got SIGAR to build. At this point I had an unsettling feeling of being alone in the vast ocean of open source. While there was evidence of others having similar problems, no one seemed to have solutions. We were on our own on this one. To make a long story shorter, after many hours of debugging with enterprise grade tools like
Purify and lots of print statements, I eventually found the root cause, fixed the problem, and patched the SIGAR library for our internal use. If you are interested in the gory details, you can read my write up of the root cause and proposed solutions
here. Essentially, the problem had to do with 64-bit cleanliness, pointer casting, and
sign extension. Since we are good citizens of the open source community,
I have submitted my proposed fixes to the developer community.
Conclusion
When I hear product folks boast about how much money they are saving by using a technology stack that’s entirely open source, I sometimes wonder if they are actually taking into account the total cost of building and maintaining their product. In the modern era of software development, part of being an award winning, world class, enterprise software company invariably entails discovering, curating, incorporating and solidifying the very best of what the world of open source has to offer. The point of this article isn’t to discourage you from using open source. Rather, it is to make you aware of the realities of open source and help you prepare for what may lie ahead. In building our platform, we’ve absorbed a lot of these costs and realities into our own internal development, QA, maintenance, and support processes. And when customers use it, they can find solace in knowing we worry about these things, so they don’t have to.
Vishal Rao is a Senior Software Engineer at Attivio and works on various parts of the core platform and add-on modules of the award winning Active Intelligence Engine. Prior to his work at Attivio, he had more than a decade of experience designing, building, and maintaining some of the most successful enterprise software products in the world like BladeLogic Server Automation, the Endeca Navigation Engine, Oracle Internet Application Server, and the Oracle Database. Image: Veerachai Viteeman/Shutterstock.com
