Today, users who want to interface with AI typically do so through a cloud-based service like ChatGPT or Microsoft Copilot, rather than on-premises.
I built a CoPilot+ AI PC (without Windows)
Part of the reason for this is that there simply aren't many good options for running AI and large language models (LLM) on end-user hardware, although we shared some ways to do this a few weeks ago. Currently, there aren't many processors with integrated neural processing units (NPUs), unless you look at the latest laptop processors from Intel, AMD and Qualcomm, or for desktops, the Ryzen 8000 series.
Lacking an NPU means users have to run AI workloads on GPUs, but this isn't perfect either. Often the graphics on the latest Intel and AMD laptop processors are sufficient, and the only other option is a dedicated graphics card, which are expensive and draw a lot of power.
However, add-on AI accelerators can be an appealing option, and Hailo is making its case with the launch of the Hailo-10 AI processor. Hailo promises to bring AI to a wide range of computers and other devices on-premises, bringing LLMs out of the cloud and to the edge.
