At TDWI: Finding a middle way between Hadoop and relational data warehousing

If you’re thinking big data analytics will solve all your BI challenges, you may be looking at it wrong…

I realized this when I was in Chicago the first week of May, attending The Data Warehousing Institute (TDWI) conference, where the theme was “Preparing for the Practical Realities of Big Data.”

There are high expectations around big data at the moment. Many people in marketing, product development, and analytics teams can’t wait to get their hands on big data intelligence to better understand and target audiences.

However, TDWI isn’t their show. At TDWI, the focus instead is on traditional database administrators, data analysts, and data scientists, and it’s a very technical conference firmly based on OLAP and OLTP analytics and hands-on issues of data warehousing.

With this in mind, I attended the keynote address by Ken Rudin, director of Analytics at Facebook—a leader in cool, cutting-edge big data processing and analysis addressing a conference of (what some would consider) old-school DBAs. The message from Rudin, who has held senior leadership positions at Zynga,, and Oracle, was fascinating: Don’t get caught in the tyranny of either/or when it comes to data analysis—businesses need both traditional relational database processing and Hadoop*-based big data analysis.

@krudin said that Facebook started out relying almost solely on Hadoop and big data when the social media giant first launched, but now is increasingly incorporating OLAP and OLTP processes into its analytics. Hadoop is best at exploring huge data sets—putting all the data into one system to discover patterns. Traditional relational data analytics is best at business, looking at focused data to derive metrics and more actionable, granular analysis. Both are valuable technologies: which one is best depends on what kind of impact you are looking for.

So the question is not, how do you get from old-school to cutting edge as soon as possible. Rather, ask which technology is right to generate impact out of data.

This was a conclusion that appealed to many at TDWI, as several people I spoke to registered a bit of skepticism about the value of Hadoop as an engine for analytics. For them, the main attraction of Hadoop is its potential to act as a backend data storage mechanism, where unstructured data can be warehoused.

Then the question becomes: What’s the best way to integrate data stored in Hadoop into a traditional OLAP or OLTP infrastructure for processing? The answer is just around the corner.

At the SAP booth, I presented a discussion of the newly released joint solution from SAP and Intel that has optimized Intel® Distribution of Apache Hadoop* software for the SAP HANA* in-memory database. Using SAP Smart Data Access* (watch for availability in coming weeks), this big data solution is able to leverage all types of data for processing in analytical applications.

And since it’s built on Intel® architecture, and leverages the full power of Intel® Xeon® E7 processors, HANA has hardware-enhanced performance and security built in, with solid-state drives and cache acceleration for blazing speed and stability. Watch this interview from TDWI to learn more about SAP HANA and how the database helps address big data challenges.

If you’re looking for the technology to get the most impact out of analytics, look no further.

Follow Tim and Twitter @TimIntel and the SAP analytics team at @SAPAnalytics.

Additionally, @TDWI has some very interesting DW feeds.

Published on Categories Archive
Tim Allen

About Tim Allen

Tim is a strategic marketing manager for Intel with specific responsibilities related to the cloud, big data, analytics, datacenter appliances and RISC migration. Tim has 20+ years of industry experience including work as a systems analyst, developer, system adminstrator, enterprise systems trainer, product marketing engineer and marketing program manager. Prior to Intel Tim worked at Tektronix, IBM, Intersolv, Sequent and Con-Way Logistics. Tim holds a BSEE in computer engineering from BYU, PMP certification and a MBA in finance from the University of Portland. Specialties include - PMP, MCSE, CNA, HP-UX, AIX, Shell, Perl, C++