We are creating data at an exponential rate. Yet, the data growth rate is not the biggest challenge for IT. The biggest challenge is that the need for useful information is growing faster than the data itself — providing a perfect storm for IT professionals and a business imperative to turn raw data into smart insights. To understand this challenge, I’d like to explore the history and evolution of big data complexity.
In 2001, Gartner analyst Doug Laney explained the initial challenges of big data in his 3Vs model. As time progressed, others have embraced the 3Vs model and incorporated two more areas of emphasis for big data analytics: veracity and value.
Ben Rossi does a nice job of articulating the impact of these five terms into one: smart data. “The purpose of smart data (veracity and value) is to filter out the noise and retain only the valuable data, which can be effectively used by the enterprise decision makers to solve business problems.”
Today, big data technology unfortunately isn’t meeting the needs of most businesses. There are two reasons. We should not be focused on the types of data, but the use case — business insights. And we must look far enough ahead in our use cases — trying to solve yesterday’s challenges and not just tomorrow’s. Michael Wu, chief scientist at Lithium Technologies, states that we are on a “maturity journey” when it comes to analytics and data visualizations. Understanding this evolution will help us better architect IT solutions today to extract the information and develop actionable insights for business decision makers.
There are three levels of analytics maturity that describe this progression:
- Descriptive analytics (what happened): A summary report of historical data, usually seen in a dashboard. Most enterprise analytics today fall into this category. An example includes a report of business data offering insights into an organization’s financials, sales, or inventory.
- Predictive analytics (what should happen): Makes predictions based on information that’s already available. An example includes financial services more accurately predicting future stock performance (noting that historical performance is not an indicator of future results).
- Prescriptive analytics (what you should do today): Analytics that not only predict the future but also deliver insights that allow you to decide today what path you should take to optimize your results. Google’s self-driving car is an example of prescriptive analytics, since the car needs to make decisions based on predictions of future outcomes. This is the use case that business leaders in a variety of industries are seeking and what’s driving the need for big data analytics.
Rossi’s concept of smart data enables intelligent insights when evaluated with a focus on prescriptive analytics. Wu summarizes nicely: “Big data technology won’t help you make bigger decisions … yet smart data can certainly help you make smarter decisions.”
Intel and Big Data Innovation
Extracting insights fast enough to support real-time business processes and decisions is critical, and companies are gathering, storing, and analyzing data they were never able to before. Intel understands the challenges and complexities facing IT professionals regarding the need to deliver high performance, cost-efficient big data solutions on a scalable, secure architecture.
As a result, Intel has joined forces with many industry leaders to enable enterprise solutions. SAP HANA, SAP Data Services, and SAP Business Objects provide solutions for real-time big data analytics using the Intel Distribution for Apache Hadoop software. Through these platforms, businesses can combine the performance of analytics with the scalability of Apache Hadoop, enabling a real-time analytics platform made to store, integrate, and analyze all business data.
Late last year, our CEO discussed a new Intel collaboration and equity investment with Cloudera aimed at bringing an enterprise-ready platform to the mainstream for impactful big data solutions.
The Big Data Maturity Journey
In summary, raw data is only useful when it’s used to add context-specific relevance, insights, and value to business operations. Smart data empowers the decision-making process by using analytics to achieve results that make sense to humans, not just machines. By making information actionable, we can make profitable decisions and solve problems in the process — and those are smart insights.