Sensor Data: Where Big Data Gets Interesting


Machines and human beings are generating some 2.5 exabytes of data each day, according to estimates from IBM. That’s 1 plus 18 zero’s worth of information daily, and it’s piling up so fast that 90 percent of the data in the world today has been created in just the last two years. Processing, storing and analyzing Big Data is driving some of the most compelling advances in technology. However, in terms of impact, Big Data is more talked about than felt in our everyday lives.

That’s about to change! Big Data from government, corporate and web sources has been available for years – information on weather, shopping patterns, Google searches, Facebook activity, and so on. Add sensor data to these analytics, and suddenly Big Data gets more interesting.

According to Harish Kotadia, in his article Big Data: The Coming Sensor Data Driven Productivity Revolution, we will begin to see big changes in industrial and business processes when we have pervasive real-time analytics of sensor data. “We are on the cusp of a major sensor data driven productivity revolution that will fundamentally change the way we do business, for the better!”

Sensor data comes from many sources – smart meters on your home utilities, traffic lights, GPS coordinates from your smartphone, TV viewing habits, security system readings, images from remote surveillance cameras, to name just a few. Information from the thousands of network-connected devices could potentially even add to this volume and variety.

Adding information from sensors injects a critical human dimension to Big Data analytics – and provides important predictive analysis for consumer behavior. Because if you’re looking for data on how people act and react in real-time situations, information from sensors is more accurate than surveys or interviews. Businesses can use this data to analyze, manage and build better products, and better understand customer behavior to differentiate and improve loyalty.

If sensor data is where Big Data gets interesting, it’s also where Big Data gets even more complex and demanding. Analyzing and storing the coming tidal wave of sensor data will require a new generation of Big Data technologies with the performance muscle to analyze masses of instantaneous sensor data and uncover insights into behavior and actions. Moving from today’s Big Data batch to real-time analytics is a natural evolution. And since this is people’s personal information we’re talking about here, Big Data platforms will require advanced defenses to protect privacy and maintain data security.


Adding sensor analysis to Big Data analytics has the potential to bring fundamental changes to how we live and do business. It will also push past the boundaries of much of today’s Big Data technology. If your organization plans to tap into the rich new insights derived from sensor analytics, it’s time to evaluate your platform and make sure you’re ready. Because a bigger, better Big Data is on the way.

Want to keep up with Big Data and enterprise computing news? Join me on Twitter. @TimIntel

Published on Categories Archive
Tim Allen

About Tim Allen

Tim is a strategic marketing manager for Intel with specific responsibilities related to the cloud, big data, analytics, datacenter appliances and RISC migration. Tim has 20+ years of industry experience including work as a systems analyst, developer, system adminstrator, enterprise systems trainer, product marketing engineer and marketing program manager. Prior to Intel Tim worked at Tektronix, IBM, Intersolv, Sequent and Con-Way Logistics. Tim holds a BSEE in computer engineering from BYU, PMP certification and a MBA in finance from the University of Portland. Specialties include - PMP, MCSE, CNA, HP-UX, AIX, Shell, Perl, C++