[caption id="attachment_6081" align="aligncenter" width="618"]
Buried somewhere in these 500 pages is the solution to your current issue.[/caption] This is the story of the comparison that just wasn't meant to be. It's a story of everything that can go wrong in the customer end of the software world, and some thoughts on what needs to be done, especially in an area known as Installers. But you've been warned: You're going to hear a few things that some Slashdot readers will likely find little more than whining. Yet the reality is this: I'm a software engineer with 25 years of experience, and for years I've wanted to point out some of the shortcomings of my own industry to help make it better for everyone involved—not only for the end-users, but also for the IT people who have to support the products; the salespeople who have to sell and later, possibly, apologize for the software; for the executives whose hands are tied because they don't have the technical knowledge to roll up their sleeves and help fix problems in the code; and for the programmers themselves who might get stuck with what some consider the absolute worst position for a programmer: maintenance of crappy code written by programmers who have long since left the organization. The story began about two months ago when my friendly neighborhood editor asked me to execute a simple task: Take two business-intelligence tools, run the same analysis, and see if you get the same answer. Sure! Simple enough, right? Now here we are, two months later, and instead of writing about the results of that analysis, I instead have a full report on the horrifying state of the software installation business. What I went through, well, should have come as no surprise to me. I've worked on huge programming teams, and I still remember back 20 years ago, seeing my employer's forum on CompuServe filled with angry messages about “General Protection Faults.” Abort, Retry, Fail? “How the hell should I know?” screamed the customers. But that was two decades. We've made a lot of progress in software development, and we don't see nearly as many crashes. Or do we? I don't know about you, but I do see my fair share of crashes. I'm using Windows right now as I type this. I kid you not. The Explorer process (the desktop) just crashed. There's a Windows Explorer window showing the files in some directory, and it's frozen. When I put my mouse over it, I see an hourglass. Other parts of Explorer aren't frozen, but there are multiple explorer.exe processes in task manager. I use an iPhone as my phone. Third-party apps crash left and right. Today the Facebook app crashed on me while I tried to click on a photo somebody posted. In none of these cases do I see a friendly error message explaining what went wrong. Is my iPhone low on memory? If so, why did it just shut down? Either way, it's a bug and the coders didn't write a proper exception handler to deal with the situation. As for my Windows machine, it has 8 Gig of RAM. I suspect the Explorer crash is due to buggy coding on behalf of Microsoft. Oh, and about this LibreOffice Writer I'm using? Occasionally it won't read the clipboard. Strange. Anyway. Onto comparing the two BI tools. They had similar problems. But in
both cases, those problems stemmed from the installation. Let's look at SAP first, then Oracle's BI suite.
SAP Installation Woes
SAP AG has a long, financially successful history. It's actually much older than a lot of people probably realize. It was started in 1972, only three years after the Unix operating system was first conceived, and the same year as the creation of the C Programming Language. In other words, this is a company with experience. Its BI tools, known as Business Objects, aren’t quite that old—but as far as analytics software goes, they’re pretty mature: Business Objects was a separate company founded in 1990, in France, and subsequently acquired by SAP for $6.8 Billion in 2007. SAP offers a free trial version of the BI tools. The downloads are massive; BI Edge is several 2-gigabyte zip files downloaded separately to make it more manageable. After you unzip them, you begin the installation process, helped along by a PDF manual. The installation itself wasn't too bad: I filled in the forms as per the manual, and eventually saw a message that everything installed correctly. Except it didn't really install correctly. One of the servers just wouldn't start up, and gave me no indication whatsoever the problem was. Another server—the Tomcat web server that hosts the user interface—started up fine, so I was able to get to a login screen through the Web browser. But using the predefined username and password was no help; the connection timed out. It was one thing after another. I dug online for help, and found many Web pages where people spoke of this same problem. One page on SAP's own forums talked about opening up the registry editor (for the Windows version) and making changes, and other people agreed this should work. (I hope I don't need to point out why that's absurd from a usability perspective.) In my case, none of the suggestions worked. I ended up with a strange error message in some log file. I found that someone else had posted about the exact same problem in the official SAP forums... and received no response whatsoever. I finally found a system log from the installation. There were a whole bunch of exception errors, but no explanation of the exception, what caused it, whether the installer took care of the problem, and if there was anything I should do to fix it. Yet the installer claimed everything had installed correctly. This went on for days. I created a new server on Amazon. I reinstalled everything. The same problem happened. At the end of my rope, I decided to change gears. Why fuss with installation? SAP has a Web-ready version where you sign up for a free trial, log in, and start using it right on their own site. Perfect. I signed up, and it worked. I was ready to go. Now all I had to do was install Oracle. I've had some experiences with Oracle, and so I boldly announced to my editor that I was finally ready to go, and I'd have the article done by the end of the week. That was three weeks ago.
Now We Have Fun with Oracle
Oracle provides a pre-built Linux image with all of the necessary business-intelligence tools already installed, running inside Oracle VirtualBox. “What could be easier?” were my famous last words. I downloaded the image and fired up the latest version of VirtualBox. It had to import the image, and after it was finished, lo-and-behold, Oracle's own Linux started up. There was a very nice PDF manual that explained the installation steps, and it looked very easy: Start eight different shells scripts by simply double-clicking their icons on the desktop. I did so, one by one. The third one ended up crashing my entire computer (not just Virtual Box). After an hour or so of digging, I discovered that I had to turn off Data Execution Prevention at the BIOS level. That's fair; I won't hold that one against Oracle. Life was still good. I fixed that little issue and all the scripts started up. My nerves were incredibly frazzled from the SAP fiasco, and so I really didn't need any more problems. I launched the browser, ready to log in, and was smacked with:
Error 500: Internal Server Error
No other explanations. Just the standard old Error 500 we all see on the web. Fine. Patience. The next day I reinstalled everything, and got the same error—even though all the scripts informed me everything installed correctly. I searched online for a solution, but this time couldn't find any answers. I dug through the logs. And what did I find? A bunch of Java exceptions, just as with SAP. I got nowhere. I tried all day and still couldn't get it to run. I wrote a profanity-laced diatribe to my editor about the problems with the software industry, and, if I remember correctly, the world in general. That's when he suggested instead of writing what I originally planned to write, why not convey my frustrations with installation? So here we are.
Whine, Whine, Whine
Realistically, nobody claims installation of enterprise-grade software is easy. There are people who do this for a living, and I would expect that some reading this article have installed either SAP or Oracle many times over. I, on the other hand, am a programmer. But let's take a step back at the bigger picture. First, I've actually been on the other side. I'm not going to say what company it was, but I worked for a large firm (now out of business) where we built enterprise-grade Java applications with probably nearly as many lines of code as SAP Business Objects. Our customers experienced installation problems that kept our tech support department busy, and we developers found ourselves constantly helping debug those issues. This often meant traveling to our client sites, which were scattered all over the planet. Clients were always losing patience, considering they were paying upwards of $2 million per installation—in addition to threatening lawsuits, more than one pulled the plug and built something in-house (while demanding we return all their payments, in the meantime). Back at our shop, management tempers would flare, and everybody ended up blaming somebody else. Looking back at what exactly went down, it was clear the problems in that organization were common to many other firms. First, management constructed a hierarchy of developers, promoting those whom they viewed as the smartest to work on architecting the new features. Below those smartest developers were midlevel people tasked with fixing the bugs in the existing code. Then came the bottom of the heap, filled with developers whom management considered the most lacking in skill-sets. (Whether they really were lacking is debatable; some of them certainly should have been looking for a new career, although a few were just on management's bad side.) The bottom of the heap was divided into two groups. One wrote in-house software that went unseen by any clients. The other group wrote... the installers. (There were also people stuck in the so-called “IT” groups where they would set up computers, install operating systems, and so on. Some of them were programmers in their own right, but often trapped and unable to escape to a development position.) That hierarchy translated into poor installers and in-house software in general. In the bigger shops, the sharpest people—who were supposed to be designing and possibly coding new features—often ended up stuck stuck doing backlogs of design documents at the request of the five or six project managers assigned to the job (and who seemed like they were more interested in creating a paper trail than creating a product). That's why the big company I worked at failed. But it doesn’t have to happen that way.
Bring on the Open Source
Compare that corporate hierarchy to the open source world. Open source development, by nature, isn't plagued with middle managers and project managers. Let's do a quick experiment: I have a server running on Amazon EC2 that I use for development purposes. It's a Quad-core server with 16 Gig of RAM. I'm going to go install MySQL on Windows right now. (I'll do some Linux experiments next.) I've already downloaded MySQL, and just need to install it. I'll be right back. It's 7:05 PM. I'm back. It's 7:18PM. It took all of 13 minutes to install MySQL Cluster (which I’ve never installed before). Now that it's up and running, I can test it by creating a database, a table inside the database, and inserting data into the table, all using the mysql prompt. If the MySQL installation is a snap, that’s how it
should be. Of course, MySQL wasn’t exactly created in the same way as other open-source projects. But the latter are generally straightforward. Ever installed Eclipse? Easy. How about Apache server? Couldn't be easier. And there are loads of really small open-source and “shareware” programs that install just fine. Now let's head over to my Linux server. First, the Linux community has perfected the whole installation process. Debian distributions use the Advanced Packaging Tool with various front-ends. I like the command-line tool myself. Installing Apache Http Server (Apache2) takes all of 15 seconds with apt-get, and maybe another couple minutes to configure if you want to add additional features. Other software is equally as easy to install. True, people do get stuck, and yes, I've installed Apache before. But for the most part, the apt-get tool does all the work for you. To be fair, there are plenty of closed-source packages that aren’t a headache to install. Microsoft Visual Studio is very easy, for example. Somebody also suggested I try out a BI tool called Kognitio, a competitor to SAP and Oracle. I launched an instance using Kognitio’s pre-built image—they provide them, to spare you the fuss over installation—and that was it: I didn't even need to log into the thing, just start the server and download the client tools to my own Windows PC. I was able to connect and start doing some data analysis without any trouble whatsoever, all within an hour. So yes, it is possible to do it right. Open source communities often have very sharp, intelligent programmers working on all aspects of projects, possibly because programmers who aren't very good also aren't interested in volunteering time with the open-source community. Meanwhile, the closed-source companies that have good installers obviously recognize the importance of devoting excellent engineers to all parts of the project, including the installers and pre-built images. In other words, the companies and groups that devote good engineers to all projects—including the installers—end up with good software everywhere.
Conclusion
We've come a long way, collectively, and software today is much easier to use than it was 20 years ago. Installers have also come a long way. But some of the giant companies continue to break the new rules. Mark my words: this article (and others complaining about the quality) will have zero impact whatsoever on the giants. These companies are so massive, they can afford to create their own rules. I'm not just talking about installers at this point. Oracle is anything but easy to use. A good friend of mine makes a lot of money as an Oracle DBA, and he’s the first to admit that. But these companies can't continue to make their own rules forever. In time, small companies will grow and create something better. Perhaps even one of you reading this article has a product you're quietly building that will make you a billionaire five years from now, and your product will be far better than what we have today. We can hope so.
