Opening up Big Data for Better Insights

Over the past few years, we’ve seen that big data is enabling fundamental changes across numerous industries, from finance to retail. But thus far, businesses, data scientists and developers have been held back from realizing the full potential of data analytics by the absence of customizable and domain-specific apps, the complexity of big data infrastructure and the lack of analytic functions with consistent application program interfaces.

At this year’s Strata + Hadoop World, we introduced the Trusted Analytics Platform, an open source analytics Platform-as-a-Service (PaaS) designed to solve these issues, making it easier for data scientists and app developers to deploy predictive models faster on a shared big data analytics platform. This enables users to harness open source development for faster, lower cost innovation.

The Intel platform is open source, allowing data scientists to easily publish data sources, data analytical pipelines, and applications. In turn, developers can use this data to build applications that range from visualization to fully articulated recommender systems using local batch or streaming data, to the benefit of every developer and data scientist in the ecosystem.

How does it work? The platform provides an end-to-end solution with three key layers:

  • Data layer that includes Apache Hadoop, Spark, and other data components optimized for performance and security

  • Analytics layer that includes a data science tool kit to simplify model development and an extensible framework to generate predictive APIs

  • Application layer that includes a managed runtime environment for cloud-native apps

More importantly, how is the platform already starting to impact the big data landscape?

First, let’s take a look at our work with Penn Medicine, who is working with Intel to advance healthcare analytics with a solution that combines patient vitals, lab records, and medications to develop predictive models that can forecast risk of disease or readmission. Using TAP, Penn Medicine will be able to build better models for predicting risk and is currently evaluating the integration of TAP with Penn Signals.

Intel is also working with Oregon Health & Science University (OHSU)’s to develop the “Collaborative Cancer Cloud,” a big data analytics solution for precision medicine that allows hospitals to securely share patient genomic data to enable potentially lifesaving discoveries. OHSU is using TAP to securely manage patient data gathered from wearable devices, labs, and surveys in a central location. The deployment takes in more than 3 million records every day with more than 360 million records already in the system. OHSU data scientists are using TAP to analyze the data to find new ways of determining a subject's overall health.

With a more open approach to big data applications and analysis, we’re creating an entirely new universe of possibilities. Greater efficiency and lower costs puts big data innovation into the hands of the many instead of the few, which will lead to never-before-seen data insights that might just change the world as we know it.