When I was young, TV news shows featured an end-of-year wrap-up of top news stories, ending with the news broadcasters and camera staff waving and wishing everyone happy holidays. I’d like to recreate that sentiment but focus on the year’s top developments in information management and big data. Here are what I see as the top trends and developments in 2013 for big data and datacenter transformation.
Traditional databases evolve to handle big data workloads
In 2013, top database providers announced re-architected legacy database technologies to better meet the challenges of big data, which requires exceptional levels of processing power to extract intelligence from growing mountains of raw data. SAP HANA* and IBM DB2 10.5 w/ BLU Acceleration Technology* combine the features of traditional row store processing with columnar data processing and in-memory columnar compression, a set of technologies that vastly speeds the scanning of massive data sets and analytical querying. Expect similar columnar in-memory compression soon on Oracle Database 12c *& Microsoft SQL Server*. To achieve the processing power required to spin through big data in search of business-changing insights and intelligence, both these solutions are optimized for Intel® Xeon® processors, which deliver higher power at a lower cost per query by making more efficient use of system resources.
Big data gets ready for the big time
In 2013, big data reached the peak of the hype cycle, but only a small percentage of organizations were actually using it to further their business goals. That’s about to change. With the release of such solutions as the Intel® Distribution for Apache Hadoop* , big data processing has left the niche of high performance computing and moved to the enterprise. With the mainstreaming of Hadoop, comes opportunity: Those businesses that have already implemented big data solutions to chart customer behaviors and sentiment will soon be in the position to gain a competitive advantage over those businesses that are slower to adapt these technologies. I expect that next year’s big data focus will start to be more on the analytics side to drive business value. Don’t get left behind!
Sensors are the big data frontier
In big data, the focus for mining useful business data has shifted from social media to sensor and machine data. In other words, the Internet of Things (#IoT)–the near endless amount of data produced by connected devices such as thermostats, cars, entertainment systems, appliances, utility meters, and other day-to-day appliances and objects. Watch our amusing animated video Surf the Big Data Wave with Intel Distribution for Apache Hadoop to learn more about impact of sensor data on big data analytics. In September, Intel announced the Intel® Quark chip, a tiny processor one-fifth the size of the Intel® Atom™ microprocessor that will lead the way toward wearable computing and low-power, high performance sensor applications for industrial, energy, and transportation markets.
Hadoop’s future is an ecosystem
Apache Hadoop was created as an open, community-developed software platform, and as it moves from niche to mainstream–with many of technology’s largest players offering distributions and services on Hadoop–comes the threat of branched development and closed environments. Hadoop is strongest and most resilient as a community-sourced technology. The Intel® Distribution remains an open source technology, and is at the center of a growing ecosystem of Intel partner solutions that offer broad choices in analytics software, networking middleware, and processing hardware. Intel honors Hadoop’s open source heritage by sponsoring such open-source initiatives as Project Gryphon, which will allow developers to deploy SQL applications on top of Hadoop, and Project Rhino, which will build the security & governance capabilities that will allow Apache Hadoop to serve as an enterprise-grade operating environment for data processing and data analytics. Keeping Hadoop an open technology helps ensure that it remains a flexible platform for triggering innovation in the broader computing environment.
Please follow Tim with the growing #BigData community @TimIntel and @IntelHadoop.