Habana Labs’ Gaudi Chip Speeds AI Training Processes 4x, Beating GPUs

The Gaudi artificial intelligence training processor was launched by a start-up last week to claims that it runs 4x faster than general processing unit-based systems.

With the introduction of the Gaudi processor and another it launched last year, the Tel Aviv-based Habana Labs directly challenges chip maker Nvidia and other leading GPU providers for market share.

Habana developed the Gaudi to train neural networksOpens a new window , which are used by engineers in perfecting the performance of algorithms by repeating processes over and over. With tens of millions of repetitions needed to train a process, anything that can speed itOpens a new window warrants serious attention by engineers.

David DahanOpens a new window , Habana’s chief executive, says the Gaudi will disrupt the AI training processor landscape. “Training AI models require exponentially higher compute every year,” he says. “So it’s essential to address the urgent needs of the data center and cloud for radically improved productivity and scalability.

“With Gaudi’s innovative architecture, Habana delivers the industry’s highest performance, while integrating standards-based Ethernet connectivity, enabling unlimited scale.”

Gaudi comes in two forms. The HL-200 is a peripheral component interconnect express (PCIe) card, an interface standard for connecting high-speed components. The card will support eight ports of 100GB Ethernet.

The HL-205 mezzanine card will support 10 ports of 100GB Ethernet or 20 ports of 50GB Ethernet. The mezzanine card also supports OCP Accelerator Module (OAM) specifications, which is an open-hardware computing accelerator module form factor that was developed by Facebook, Microsoft and Baidu.

Additionally, Habana is selling the HLS-1 system, which includes eight Gaudi processors for data center scaling.

Recording-Breaking Performance

In internal tests of Gaudi’s speed, Habana reports running it against Nvidia’s main V100 data center graphics card. Habana says the Gaudi ran 3.8x  faster than the Nvidia chip on the ResNet-50 networkOpens a new window , setting new industry performance records. The ResNet-50 is a training network commonly used in the AI community for setting benchmarks.

Last year Habana launched the Goya AI Inference Processor. It has demonstrated high performance, power efficiency and real-time latency, and it also beats ResNet-50 records. It is designed to power AI models that have already been trained and are used in production to process live data.

“With its new products, Habana has extended from inference into training, covering the full range of neural-network functions,” says Linley Gwennap, principal analyst at the Linley Group. “Gaudi offers strong performance and industry-leading power efficiency among AI training accelerators. As the first AI processor to integrate 100G Ethernet links with RoCE support, it enables large clusters of accelerators built using industry-standard components.”

At Facebook, technology director Vijay Rao says the social media company “is pleased that the Goya AI inference processor has implemented and open-sourced the backend for the Glow machine learning compiler and that the Habana Gaudi AI training processor is supporting the OCP Accelerator Module (OAM) specification.”

A Customizable Processor

Users can program and customize the Gaudi processor. It has a second-generation Tensor Processing Core cluster, as well as development tools, libraries and a compiler. Habana’s SynapseAI software stack consists of a kernel library and open toolchain that allow customers to add proprietary kernels.

In addition to its Tel Aviv headquarters, Habana maintains offices in San Jose, Beijing and Gdansk and employs about 150 people. The startup was founded in 2016.

In its series B funding round in November 2018, Habana attracted $75 million from investors led by Intel, as well as Bessemer Venture Partners and Battery Ventures.

The technology market intelligence group CB Insights in February included Habana Labs in its top 100 AI companies to watchOpens a new window .

While the price of the new processor has not yet been announced, it will be competitively priced, the company says. Habana is planning to sample Gaudi platforms with select customers throughout 2019.

At a time when the global AI market is burgeoning with analysts valuing it at $7.35 billion in 2018, Gaudi’s efficient credentials mean it is certain to attract serious interest from the technology customers.