Deci delivers breakthrough inference performance on Intel's 4th Gen Sapphire Rapids CPU

The Intel-Deci breakthrough enables AI developers to achieve GPU-like AI inference performance on CPUs in production for both computer vision and NLP tasks

TEL AVIV, Israel, Jan. 25, 2023 /PRNewswire-PRWeb/ -- Deci, the deep learning company building the next generation of AI, announced a breakthrough performance on Intel's newly released 4th Gen Intel® Xeon® Scalable processors, code-named Sapphire Rapids. By optimizing the AI models which run on Intel's new hardware, Deci enables AI developers to achieve GPU-like inference performance on CPUs in production for both Computer Vision and Natural Language Processing (NLP) tasks.

Deci utilized its proprietary AutoNAC (Automated Neural Architecture Construction) technology to generate custom hardware-aware model architectures that deliver unparalleled accuracy and inference speed on the Intel Sapphire Rapids CPU. For computer vision, Deci delivered a 3.35x throughput increase, as well as a 1% accuracy boost, when compared to an INT8 version of a ResNet50 running on Intel Sapphire Rapids. For NLP, Deci delivered a 3.5x acceleration compared to the INT8 version of the BERT model on Intel Sapphire Rapids, as well as a +0.1 increase in accuray. All models were compiled and quantized to INT8 with Intel® Advanced Matrix Extensions (AMX) and Intel extension for PyTorch.

"This performance breakthrough marks another chapter in the Deci-Intel partnership which empowers AI developers to achieve unparalleled accuracy and inference performance with hardware-aware model architectures powered by NAS," said Yonatan Geifman, CEO and Co-Founder of Deci. "We are thrilled to enable our joint customers to achieve scalable, production grade performance, within days".

Deci and Intel have maintained broad strategic business and technology collaborations since 2019, most recently announcing the acceleration of deep learning models using Intel Chips with Deci's AutoNAC technology . Deci is a member of the Intel Disruptor program and has collaborated with Intel on multiple MLPerf submissions. Together, the two are enabling new deep learning based applications to run at scale on Intel CPUs, while reducing development costs and time to market.

If you are using CPUs for deep learning inference or planning to do so, talk with Deci's experts to learn how you can quickly obtain better performance and ensure maximum hardware utilization. To learn more about the Deci-Intel collaboration, go to https://deci.ai/intel/

About Deci

Deci enables deep learning to live up to its true potential by using AI to build better AI. With the company's deep learning development platform, AI developers can build, optimize, and deploy faster and more accurate models for any environment including cloud, edge, and mobile, allowing them to revolutionize industries with innovative products. The platform is powered by Deci's proprietary automated Neural Architecture Construction technology (AutoNAC), which automatically generates and optimizes deep learning models' architecture and allows teams to accelerate inference performance, enable new use cases on limited hardware, shorten development cycles and reduce computing costs. Founded by Yonatan Geifman, Jonathan Elial, and Professor Ran El-Yaniv, Deci's team of deep learning engineers and scientists are dedicated to eliminating production-related bottlenecks across the AI lifecycle.

Media Contact

Garrett Krivicich, Deci, 1 786-233-7684, garrett@headline.media

SOURCE Deci