The Evolution of Big Data Use at Intel


Since Intel IT generated US$351 million in value from Big Data and analytics during 2014, you might wonder how Intel started on the road to reach that milestone.  In this presentation named “Evolution of Big Data at Intel: Crawl, Walk and Run Approach” from the 2015 Hadoop Summit in San Jose, Gomathy Bala, Director, and Chandhu Yalla, Manager and Architect, talk about Intel IT’s big data journey. They cover its beginning, current use cases and long term vision.  Along the way, they offer some useful information to organizations just starting out to explore big data techniques and uses.

One key piece of advice that the presenters mention is to start on small, well-defined projects where you can see a clear return.  That allows an organization to develop the skills to use Big Data with lower risk and known reward, part of the “crawl” stage from the presentation title.  Interestingly enough, Intel IT did not rush out and try to hire people who could immediately start using tools like Hadoop.  Instead, they gathered engineers who were passionate about new technology and trained them to use them.  This is part of “walk” stage.  Finally, with that experience, they developed an architecture to use Big Data techniques more generally.  This “run” stage architecture is shown below, where all enterprise data can be analyzed in real time.  We will be talking about Intel's Data Lake in an upcoming white paper.

Another lesson is to evaluate Hadoop distributions and use distributions that is core open source. This is one of a number of criteria that were established.   You can see more on Intel IT’s Hadoop Distribution evaluation criteria and how we migrated between Hadoop versions in a previous blog entry.

A video of “The Evolution of Big Data at Intel, Crawl, Walk and Run Approach” can be seen here, and the presentations slides are available as a slideshare.  A video of Intel CIO Kim Stevenson talking about Intel's use of Big Data is shown in the presentation video, but a clearer version can be found here.