In 2011, the amount of information created and replicated will surpass 1.8 zettabytes (1.8 trillion gigabytes)—growing by a factor of nine in just five years. While 75 percent of the information in the digital universe is generated by individuals, enterprises have some liability for 80 percent of information in the digital universe at some point in its digital life. The number of ‘files,’ or containers that encapsulate the information in the digital universe, is growing even faster than the information itself as more and more embedded systems pump their bits into the digital cosmos. In the next five years, these files will grow by a factor of eight, while the pool of IT staff available to manage them will grow only slightly.
This information, which comes from a June IDC study, highlights the fact that going forward, big data management is going to be a literal and figurative growth industry. (Dice has more than 200 job listings in the big data field today.)
One job listing outlines these responsibilities:
- Provide technical and team leadership for all projects that involve data and data architecture
- Act as the subject matter expert analyzing the latest and greatest technologies to solve problems associated with business intelligence, analytics, data visualization, etc.
- Define and document architectural standards
- Serve as a leader to more junior staff, provide insight to other functional teams, including executives
This is heady stuff. A position such as Big Data Architect will require some serious data management experience. If you have it, you’re going to be in demand.
As we’ve noted before, “Big data really is different from the data warehousing, data mining, and business intelligence analysis that have come before it. Data is being generated with greater velocity and variability than ever before, and, unlike data in the past, most of it is unstructured and raw (sometimes called ‘gray data’).”
Do you have what it takes to become a big data guru? Look into it. You’d likely prosper as a pioneer in what McKinsey calls “the next frontier for innovation.”