How an NPU powers the current AI boom, from phones to PCs

intel core ultra NPU
Image: Intel.

A specialised chip optimised for accelerating algorithms, a neural processing unit, or NPU, is designed to supercharge devices that take advantage of artificial intelligence. Although relatively new to the world of desktop computing, the technology has been around for a while in various forms.

Originally, graphics processing units (GPU) were created to accelerate graphics and ease the load on a device’s central processing unit (CPU). Now, NPUs are optimised to accelerate the algorithms that underpin artificial intelligence and machine learning.

Devices can hand over demanding new AI-based tasks to the NPU. As well as performing these tasks faster, the NPU also takes the load off both the CPU and GPU in order to boost overall device performance and power efficiency. 

The birth of the NPU

Also known as AI accelerators or deep learning processors, early NPUs were designed for dedicated AI hardware, for example aiming to replicate neural networks in an effort to tackle complex computing challenges.

As AI became used more widely, cloud computing giants adopted NPUs in their servers to handle the growing demand.

While Nvidia GPUs dominate this space, the cloud computing giants have also developed their own server NPUs such as Google’s Tensor, Amazon’s Trainium and Inferentia, Microsoft’s Azure Maia 100 AI Accelerator and Meta’s MTIA.  

Neural processing units in your hands

As AI becomes more a part of our everyday lives, neural processing units have come to smartphones and tablets.

A wide range of chipmakers including Intel, AMD, Qualcomm, Samsung, Google and Apple build NPUs – or build specialised NPU cores into their System-on-a-Chip (SoC) designs.

These include Apple Bionic and M-series chips in Macs, iPhones and iPads, Exynos chips powering Samsung Galaxy devices, Tensor chips powering the Google Pixel range, Huawei’s Kirin chips and Qualcomm’s Snapdragon chips.

Mobile NPUs underpin a wide range of new AI-powered features such as speech recognition, smart assistants and generative AI. They also improve processing-intensive graphics features like video stabilisation, upscaling, background blurring and photo/video editing with object detection.

Along with taking the load off the mobile CPU and GPU, these mobile ​​NPUs also reduce the AI load in the cloud. Handling more AI tasks on-device offers a speed boost, while also improving privacy by ensuring less personal data is sent to the cloud.

Desktop NPU surge

Desktop chips are also embracing neural processing units in order to make the most of AI.

Rather than relying on a separate card, similar to a discrete high-end graphics card, NPUs are integrated into the main desktop chip. Examples include Intel’s Core Ultra ‘Meteor Lake’ chips and AMD’s Ryzen 8000-series chips.

AMD Ryzen 8040 AI laptop processor
Image: AMD.

Intel Core Ultra devices are already coming to market, such as HP and Lenovo‘s new laptop ranges.

Some examples Intel showcased that take advantage of the NPU’s capabilities included powering a local AI desktop assistant, image generation and smoothing video editing. Generative AI is also coming to the desktop with Microsoft’s Copilot AI desktop assistant, bringing generative AI features to everyday Windows 11 applications.

As more devices include neural processing units, expect to see the use cases of AI technology grow exponentially.

Read more computer tech news on GadgetGuy