Intel IT’s Big Data Journey

Big Data is a relative term and means different things to different enterprises.  Intel IT defines big data as “data sets that are unmanageable using traditional technologies” – this unmanageability stems from exploding volume, variety, and velocity of data. Up to now, it has been technologically impossible and/or not cost effective to deal with data at this level.

Mining and analyzing big data gives us deeper and richer insights into business patterns and trends, helping drive operational efficiencies and competitive advantage across Intel—in manufacturing, sales and marketing, product design, and information security. We are investing in advanced analytics, smart infrastructure for big data, and self-service business intelligence (BI) capabilities to solve high-value business problems and gain actionable insights in near real time.

The cost to store data is now significantly less than the value it represents.  Thanks to Moore’s law the cost to store and analyze big data has reduced.  At the same time, data growth has exploded because of both user and machine generated data from the dramatic growth of the internet.  Through big data, we can mine large data sets to see the bigger picture and make smarter and faster decisions.  Applying predictive analytics on top of that, we can anticipate the future, act proactively, and change the future.

We were chartered to function like a big data service provider and evaluate multiple Hadoop distributions and deliver a fully integrated production platform to our business groups to extract value from big data.  Our 3 primary goals were to:

  • Perform cost-effective BI in an era of big data to enable competitive advantage 
  • Provide the right data in the right form to the right people at the right time
  • Solve high-value business problems with actionable insights in near real time

·       In 2010, we standardized and centralized the collection, storage, processing, and distribution of the information Intel needed to run its business.  This enabled us to make more effective and extensive use of BI tools, drive Operational Efficiency, and ultimately deliver competitive advantage for Intel.

·      To provide cost-effective BI in an era of big data, required us to move from a one-size-fits-all enterprise data warehouse to a multi-DW BI strategy which has proven to be a more cost-effective way to better address our business requirements.  Already we’re seeing a cost avoidance of nearly $250k in the first year alone.

·       From there, we had to figure out data virtualization so that we could integrate our containers and not inefficiently replicate data across multiple containers.

·       And from there we have a team of very skilled data scientists that are developing predictive algorithms and machine learning capabilities.

·       Lastly, a key part of our strategy is to enable and deliver SSBI, Visualization, and mobile BI because at the end of the day, a human has to be      able to act on this information and make smarter and faster decisions for our business. 


F     Follow me on twitter @ajayc47