The Cutting Edge of AI… Everything Matters!

Running a marathon in under two hours requires shoes engineered for the task. AI is no different. 

Growth of data is exponential. In five years, humans & machines will produce 10x more than we did this year. And more than 70% will be created at the Edge. Only half will move to public clouds—the rest will be processed, stored and analyzed at the Edge.

Why? Several reasons…bandwidth availability & cost, data sovereignty, security and the increasing need for real-time processing. Processing that volume of data, at that velocity, requires a new approach…an approach that is significantly more efficient than current options. Which is why we are creating purpose-built products & tools for our most demanding customers.

AI is rapidly being adopted at the Edge by companies of every size, across every industry—from Smart Cities to Industrial to Retail and Health & Life Sciences. Our Edge portfolio has been tremendously popular with customers and developers because it makes the existing infrastructure, investments, and experience they have on Intel more valuable.

Capturing the full benefit of AI requires making software open and easy, designing products & tools for the unique demands of the Edge, and investing in the next generation of talent.

Earlier this week at the Intel® AI Summit, I spoke about hardware and software innovations that not only accelerate Edge AI performance further, but do so while making performance easy to attain. We disclosed our next-gen Movidius Vision Processing Unit (VPU), codenamed Keem Bay. Keem Bay is a low power (4-15 watts), high-performance Edge Inferencing product purpose-built for Deep Learning, Vision & Media. This is the work of over 1,000 incredibly talented people from ten teams across seven countries. We have early customers testing it and will launch in the first half of 2020.

This architecture is highly optimized for Edge inference, with ground-breaking leaps forward in performance. And it’s a workhorse. With flexible form factors, customers can adopt it chip-down for cameras, on compact M.2 cards for things like kiosks and robotics, all the way up to full-power PCIe cards that will be able to run multiple VPUs in parallel for high-density, scalable Edge AI server acceleration. This will enable the full range of Edge experiences our customers are asking for.

Compared to alternatives in the market, Keem Bay really shines. In early testing, KMB is fast—up to 4X the performance of NVIDIA’s comparable TX2. Nvidia’s Xavier part is actually a tier above KMB—30W and 5x the size—but KMB delivers there as well, being on par1 with raw performance @ only 1/5th power.

And that’s the key…at the Edge, raw performance is only part of the equation—customers also care about power, size and latency. Keem Bay shines here as well. From a power point of view, Keem Bay is “green”—delivering over 6x the inference performance per watt over NVIDIA’s TX2.

And it’s small: measured by inferences per mm2, Keem Bay is 8.7x the TX2; with Xavier also far behind.

And finally, it’s efficient, because it is engineered for Edge Inference. There are no wasted TOPS, with 4x the inferences per second per TOPS than NVIDIA’s Xavier.

Rather than taking products designed for another purpose, we are engineering specifically for Edge Inference.

Just as Nike's Vaporfly helped Kipchoge break the 2 hour marathon record, purpose-built silicon will enable a new wave of AI at the Edge.

But it’s not just about silicon innovation—we believe in democratizing AI. Easy-to-use tools like our open source OpenVINO™ toolkit are key to a level playing field.

OpenVINO is a dev tool & runtime to enable customers to write once and deploy across a wide variety of Intel silicon at maximum performance. With OpenVINO, highly performant systems are accessible to people who don’t need to understand hardware architecture and electrical engineering.

We also launched a new companion to OpenVINO—the Intel Dev Cloud for the Edge. Here’s how it works:

First, using their favorite standard frameworks, developers run their model through the OpenVINO model optimizer to accelerate for the Edge on any Intel compute, automatically.

Next, because developers have asked how to know which Intel product will provide the right performance for their deep learning model use case, Dev Cloud for the Edge lets them test their models for free against the full range of Intel Edge silicon. It informs their hardware landing zone decision and lets them test before they buy. Over 2,700 customers have been using the beta—and loving it.

When they are ready to deploy, the OpenVINO runtime takes into account the capabilities of the full system and can allow customers to achieve automatic, software-defined parallelism by load balancing across CPU cores, iGPU, accelerator, or other hardware.

Building a solution or algorithm is only one step—scaling a solution or algorithm to the Edge takes an ecosystem of partners, and our AI: In Production program pairs solution developers, ODMs, System Integrators and more to enable easy deployment, at scale.

Oh, and one more thing…for too long deep learning courseware has been focused on AI in the cloud. We announced this week a partnership with Udacity to launch an Edge AI Nanodegree that will train the next generation of developers on how to do AI where the data is generated: at the Edge. We’ll be awarding 750 scholarships for this course.

So, key takeaways:

  • Keem Bay’s groundbreaking architecture will deliver superior raw performance and efficiency when it launches 1H’20, at a fraction of the power, a fraction of the size & a fraction of the cost.
  • It complements our full portfolio of products, tools and services purpose-built for the Edge.
  • This portfolio—including OpenVINO and our new Dev Cloud for the Edge—makes AI accessible to everyone, not just the experts and companies with fleets of data scientists.
  • And they can get started with the first Edge AI nanodegree we’ve launched with Udacity.
  • Each of these products individually demonstrate extraordinary performance and value to our customers. When employed collectively, they are an unparalleled set of resources that aim to make Edge AI available to every customer, at every size, across every industry without massive investments in talent or capital.

Cheers—

JB


1 Keem Bay throughput within 10% vs Xavier throughput.

Product Intel 3rd Gen Intel Movidius VPU “Keem Bay” NVIDIA Jetson TX2 Huawei Atlas 200

(Ascend 310)

NVIDIA Xavier AGX
Testing as of 10/31/2019 10/30/19 8/25/19 10/22/19
Precision INT8 FP16 INT8 INT8
Batch Size 1 1 1 1
Product Type Keem Bay EA CRB Dev kit (preproduction) Jetson Developer kit Atlas 200 Developer kit Jetson Developer kit
Mode N/A nvpmodel 0 Fixed Freq N/A nvpmodel 0 Fixed Freq
Memory 4GB 8GB 8GB 16GB
Processor ARM A53 x 4 ARM v8 Processor rev 3 (v8l) × 4 ARM A53 x 8 ARM v8 Processor rev 0 (v8l) × 8
Graphics N/A NVIDIA Tegra X2 (nvgpu)/integrated N/A NVIDIA Tegra Xavier (nvgpu)/integrated
OS Ubuntu 18.04 Kernel 1.18 (64-bit) on Host

Yocto Linux 5.3.0 RC8 on KMB

Ubuntu 18.04 LTS  (64-bit) Ubuntu 16.04 Ubuntu 18.04 LTS  (64-bit)
Hard Disk N/A 32GB 32GB 32GB
Software Performance demo firmware JetPack: 4.2.2 MindSpore Studio, DDK B883 JetPack: 4.2.1
Listed TDP N/A 10W 20W 30W

The above is preliminary performance data based on pre-production components.

Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors.

Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more information go to www.intel.com/benchmarks.

Performance results are based on testing as of Oct 31, 2019 and may not reflect all publicly available security updates. See configuration disclosure for details.  No product or component can be absolutely secure.

Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. Check with your system manufacturer or retailer or learn more at www.intel.com.

Intel, the Intel logo, Xeon™ and Movidius™ are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others.

© Intel Corporation.

Published on Categories Internet of ThingsTags , , , , , , ,
Jonathan Ballon

About Jonathan Ballon

Jonathan Ballon, Vice President, Internet of Things Group, is passionate about the application of technology towards our environment, human productivity, safety, education, health & longevity. At Intel, Jonathan is responsible for a global team chartered with driving and accelerating innovation and growth in various market segments. His team is a pioneer in artificial intelligence and deep learning applications with computer vision capabilities, supported by software tools & a robust ecosystem. Additionally, Jonathan leads the IoT channel routes-to-market and is responsible for Intel’s China engineering teams and business.