I have been exploring practical ways to integrate local AI models into daily workflows, specifically using hardware that is not necessarily high-end. A great use-case are Macs and iPhones that have “Apple Intelligence” enabled. Apple’s built-in models are accessible for just about any task through the command line, Shortcuts and other tools.
In my latest video, I show you how to put these models to work!
Apple’s “Foundation” models are integrated into macOS, iOS, and iPadOS. Their primary advantage is efficiency. On a machine with limited memory, large external models can be difficult to run alongside other applications. These system-level models spin up quickly and maintain a minimal memory footprint. They allow for text generation and data processing without requiring the user to close other programs or interrupt their work. While they are not as fast as cloud-based alternatives, they provide a stable environment for local tasks.
To interact with these models beyond basic text editing, I use a free application called Locally, which is available on the App Store for Mac, iPhone, and iPad. This app provides a direct interface for the foundation models and allows for tasks such as PDF summarization and other AI chat activites. Although the context window on older devices is limited to approximately 4,096 tokens, the model is capable of extracting key points and specific data from documents.
Using these models ensures that sensitive or confidential documents never leave the device, providing a layer of privacy that cloud models cannot offer. On newer hardware, such as the M3 Mac or recent iPhones, Apple utilizes the “Advanced” model—a 20-billion parameter mixture of experts—which offers more capability while remaining resource-efficient.
Advanced users can also access these models via the terminal in the latest version of macOS. By using the command line, it is possible to initiate chats or process files within a specific folder. I tested this by having the model analyze photographs of receipts. Despite the images being folded or having faded text, the model’s optical character recognition capabilities allowed it to identify the merchant, the date, and the total transaction amount and drop the data into a spreadsheet.
The Shortcuts app offers a more automated approach. I created a shortcut that uses the system’s foundation model to take a photo of a receipt on my iPhone, extract the relevant data, and append that information to a CSV file stored in my iCloud Drive. Because Shortcuts syncs across the Apple ecosystem, a receipt photographed on an iPhone can be processed locally on the phone, with the resulting data appearing on a Mac almost instantly. While the final file may sit on a cloud drive, the actual intelligence used to read and interpret the receipt remains on the device.
To further test the limits of these local tools, I used a frontier model to help write a custom Mac application to allow for bulk processing of receipts. While the coding process required an external model, the resulting application runs entirely locally. It allows for batch processing through a simple drag-and-drop interface, using the Mac’s internal model to categorize expenses and record totals. Even on older hardware, the processing is relatively fast and the accuracy is high enough to be useful for basic expense tracking.
Apple’s Foundation models are trained for the specific types of tasks people perform on their computers and phones, such as scheduling, summarization, and data organization. They function as quiet, efficient neighbors to the other software on the system, and they represent a powerful tool that is already available to most users without any additional installation or complexity.
The most effective way to understand the capabilities of these built-in assistants is through direct experimentation. Let me know what you come up with!