Intel AI in Action at the World’s Biggest Cloud Service Providers

I’ve been on a nice summer break spending time with family and trying my best to resist the urge to check work emails. Now, I am back to work feeling rejuvenated. The Intel team has been busy!  In just a couple of months, they’ve ushered in the summer of AI with major advancements with our leading cloud service providers— the most recent being the Microsoft Project Brainwave announcement at Hot Chips 2017. In this post, I want to give you a little more insight about Project Brainwave and other breakthrough IA based AI cloud offerings across the globe.

Intel Helping to Elevate Microsoft’s AI Leadership

Microsoft has been using Intel field-programmable gate arrays (FPGAs) to improve the performance and efficiencies of Bing and Azure for the last few years. This summer, when we launched our new Intel® Xeon® Scalable processors, Kushagra Vaid, GM of Azure Hardware Infrastructure, Microsoft’s Azure Platform, touted Microsoft’s vision of AI and analytics coming together to power the next generation of enterprise applications, and that the combination of Xeon and Intel FPGA is extremely powerful to solve the challenges they see with AI.

Project Brainwave, Microsoft’s new deep learning platform unveiled at Hot Chips on August 22 represents a major leap forward in both performance and flexibility for cloud-based serving of deep learning models. Project Brainwave is keenly focused on AI inference to deliver real-time AI by design. As cloud infrastructure is being widely used for live data streams, whether they be search queries, videos, sensor streams, or interactions with users, real-time AI becomes increasingly important. Microsoft selected Intel® Stratix® 10 FPGAs to ensure the system processes requests as fast as it receives them, delivering 39.5 Teraflops of achieved performance at ultra-low latency of less than one millisecond.

The World’s AI Inference Runs on IA

Beyond Microsoft, many other Cloud Service Provider (CSP) customers are relying on Intel Architecture for their AI inference workloads.

At Amazon Web Services (AWS), the computational power of the Intel® Xeon® Scalable processor is enabling AWS customers to use more data to create innovative new products and experiences powered by machine learning. Together with Intel, AWS has optimized its deep learning engine with the Intel Math Kernel Library (MKL) over Xeon Scalable for their C5 instances, resulting in more than 100X inference performance enhancement.

Across the Pacific Ocean,, China’s 2nd largest online retailer, is investing heavily in AI and drone delivery technologies. Recently, using Intel®Arria®10 FPGA, implemented Convolutional Neural Network (CNN) and Long Short Term Memory Network (LSTM) to help analyze massive amounts of images on for Optical Character Recognition(OCR). The test results show that the Arria®10 FPGA can achieve 5x improvement in LSTM performance acceleration compared to GPU alternatives, avoiding some inefficient calculations and reducing image recognition latency[1].

Expansion into AI Training

Intel’s AI solutions spans beyond inference, we are enjoying healthy growth in AI training as well.

First, Intel Xeon Scalable Processors can deliver scalable machine learning with attractive TCO, and ease of implementation and management.

Google was the first to launch Xeon Scalable based cloud instances on Google Cloud Platform (GCP) in Feb of 2017. Since then, many customers running compute-intensive workloads such as AI/machine learning have seen significant benefits. An example is Descartes Labs, a GCP customer who applies AI to satellite imagery for applications such as crop yield forecast, achieved >2.5X cost reduction by moving to Intel Xeon Scalable processor based GCP instances.

In addition, an Intel initiative called BigDL promises to quickly bring machine learning into the mainstream by enabling deep learning applications to piggyback on familiar Xeon processor based infrastructure. This takes advantage of familiar data architectures such as Apache Hadoop or Spark that enterprises and CSPs already put in place for big data analytics. Major CSPs, such as Microsoft, Amazon, and Alibaba, are rolling out BigDL as part of their services to speed adoption of deep learning and enable their customers to extract even more business value from their data.

Last but not least, Intel® Xeon Phi™ Processors is gaining market traction. Meituan, the world’s largest online and on-demand delivery platform (you can think of it as China’s Groupon, Yelp and Booking combined), recently deployed Intel® Xeon® Scalable processors and Intel® Xeon® Phi to run its AI workloads for both internal applications and AI-as-a-Service in its public cloud - Meituan Open Platform (MOS).

Furthermore, the Chinese search giant, Baidu, investigated Xeon Phi as a processor option for deep learning. They concluded that compared to GPU alternatives, Xeon Phi also has tremendous floating point capability, while having potential advantages in the programmability, memory model, and networking characteristics.

I’m excited about the future of IA in AI and all the possibilities for businesses to create new services, save on TCO and grow. Intel Architecture is powering the world’s AI clouds, and I encourage you to learn more about AI, Intel’s AI offerings, and our success in AI at