Apple's Hardware Is Ready for On-Device AI and PrismML Just Delivered a Real Breakthrough

On July 14, a start-up with Caltech roots, PrismML, released Bonsai 27B, a free artificial intelligence (AI) model compressed enough to run locally on an Apple (NASDAQ: AAPL) iPhone 17 Pro. According to the company, it retains roughly 90% of the original model's performance.

A capable AI small enough to run on an iPhone is great news for consumers. PrismML's CEO recently told CNBC that Apple is in "very early" discussions about the technology. In time, free, local, open-weight AI could become an à la carte menu for consumers, and Apple's own silicon was designed to run them.

Image source: Getty Images.

A serious model that finally fits on a phone

A traditional 27-billion-parameter model cannot fit in a phone's usable memory. At 3.9 gigabytes, Bonsai is built on Alibaba's open-weight Qwen3.6 and can run on an iPhone 17 Pro, as well as on iPad, Mac, and PCs.

Shrinking open models is not new, but compressing one of this class while retaining its performance is. For Apple, whose latest chips were designed to run AI on-device, this could raise the stakes for its internal development of foundation models.

The A19 and A19 Pro chips in the latest iPhones feature neural accelerators, which the company says provide a significant boost to AI performance. On the company's second-quarter earnings call in April, management described the Mac as "the best platform for AI," with its silicon capable of running advanced AI like never before.

Who's going to monetize consumer AI?

Most consumer AI usage is free. OpenAI surpassed 900 million weekly active users in February, with roughly 50 million paying subscribers and a conversion rate of around 5.5%. To the extent the frontier labs counted on consumers paying for AI inference, that opportunity may shrink as open-weight alternatives continue to improve and take share.

Apple's position is different. Its business is selling devices, so a local-AI wave means more memory and capacity, which raises the cost of goods sold for the hardware maker.

Its own AI execution has been lackluster so far. The previously delayed and rebuilt Siri fell short in internal testing in February, though management sounded optimistic on its July 30 earnings call. Siri AI has been in public beta testing for a few weeks, and the feedback has been positive.

In July, Apple sued OpenAI in federal court, alleging trade-secret theft tied to former engineers who joined the lab, including claims that trade secrets were taken to help OpenAI build its own devices.

Apple's lawsuit against OpenAI puts the consumer AI fight front and center. As expected, the company remained silent about the lawsuit during its earnings call.

Apple has the silicon. With breakthroughs such as PrismML's, consumers may not have to wait much longer for capable AI that can run offline and keep data on the device without a monthly fee.

Bryan White has no position in any of the stocks mentioned. The Motley Fool has positions in and recommends Apple. The Motley Fool recommends Alibaba Group. The Motley Fool has a disclosure policy.