Business Intelligence Systems: Turning Data into Information and empowering Decision Makers

Businesses produce and capture incredible amounts of data. Sales figures pour in from one system, purchasing numbers come in from another, and HR metrics roll in from yet a third. Out of these vast and disjointed streams, executives are expected to gain the insight and context needed to not only run their business day to day, but develop the strategies that will steer the business toward continued success. The technology professionals working in the business intelligence sector help provide that insight by leveraging technology to turn raw data into usable information.

Business intelligence (BI) is about distilling and presenting relevant and timely data to the end user for analysis and action. By creating systems that gather the appropriate slices of data from disparate sources, BI provides decision makers with the tools to sift though mountains of facts and measurements to find actionable meaning. Using historical data, business leaders can use BI systems to measure the achievement and current health of their company, or look to the future using predictive analytics. If you¿ve ever had Amazon make a particularly prescient suggestion based on your historical buying habits, you¿ve seen the power of the predictive analytics side of BI.

The Landscape

After several rounds of buyouts and consolidations in the industry over the last few years, the biggest players in this market are Oracle, SAP, IBM, and Microsoft. Many BI systems are implemented using software products from one of these vendors, or a combination best-of-breed, using a mix of solutions from the big four. Each of the major software vendors, and smaller tier providers as well, have strong consulting and services divisions that employs hundreds of BI professionals.

While the recession hasn¿t hit the BI sector as heavily as other areas of IT, the job market has softened. The most recent Magic Quadrant for Business Intelligence Platforms report from researcher Gartner offers some reasoning behind this trend saying, "BI platform revenue will be less affected by the economic downturn than some other technologies because of the heightened need to make better, fact-based decisions  –  BI is a vital competitive tool of increased importance in an environment where doing business more smartly, in order to maximize share of the reduced revenue in circulation, is a necessity."

While the report goes on to caution that the double digit growth of 2008 could cool to about 7 percent in 2009, it predicts growth will continue steadily through 2012. As businesses implement BI systems, there will be steady demand for the analysts, developers, and data warehouse professionals that are involved with both large and small implementations.

In addition to the steady maintenance of the early BI adopters in national restaurant chains and large retailers, there are two new bright spots for BI: Both the healthcare and energy industries are currently investing in large intelligence initiatives. While not recession-proof, organizations in these areas currently have the revenue to spend, and an increasing need for BI systems.

For fresh college graduates, offshoring has been a troubling trend in BI over the last few years as lower-level development and design tasks have been outsourced overseas. "What’s happening is that the required work is being done in other countries," says Adrian Ward, a principle consultant at Rittman Mead Consulting. "We have a set of companies that are competing very hard, and therefore, they are looking at the lowest cost options."

Companies are also trending away from consultants toward full time employees. "Some of the trends right now are that organizations that have money to invest in BI are rolling off their consultants and bringing on full-time employees because it’s much more cost effective," says Matt Mueller, president of CBIG Recruiting and Staffing in Chicago. "Instead of paying a consultant at $100 an hour, or in many cases a lot more, they can find someone at $100,000 and save themselves $60,000 – $70,000 per year."

Roles and Career Paths

Jobs in BI fall into 3 categories

  • Analysis
  • Data Warehouse
  • Reporting / Presentation

In analysis, the business analyst’s role is to walk between the worlds of business and technology. Analysts interview the business domain experts to gather the business requirements that need to be met. They then write functional specification documents that the technical teams use to design and construct necessary solutions.

Data Warehouse / ETL 

Individuals working in the data warehouse are involved in building, populating, maintaining, and managing data structures and databases. Data architects and modelers design both relational and multidimensional structures to accomplish the goals set out in the technical specification. Developers generally work in the ¿Extract Transform and Load (ETL) portion of the data warehouse. They create systems to extract data from different sources, transform it into the desired format, and load it into the data warehouse. 

Reporting / Presentation

The BI professionals working in reporting and presentation create the tools that allow the business decision makers to consume the data in a format that has context for them.  Developers working in this area work with analysts and business domain knowledge experts to create dashboards, scorecards, and reports from the extracted data residing in the data warehouse. Other professionals in this area include report developers, reporting analysts, and BI Architects.

Skills and Qualities

  • Technical aptitude
  • Strong database and data model knowledge
  • Ability to understand and speak the language of business
  • Knowledge of financial accounting procedures and standards
  • Excellent oral and written communication
  • Strong programming ability