Apple Intelligence
Apple Intelligence is an artificial intelligence platform developed by Apple Inc.[1] Relying on a combination of on-device and server processing, it was announced on June 10, 2024 at WWDC 2024 as a feature of Apple's iOS 18, iPadOS 18, and macOS Sequoia operating systems, which were announced alongside Apple Intelligence. Apple Intelligence will be free for all users with supported devices. It is scheduled to enter into a developer beta in the United States in late 2024 and will fully launch in 2025.[2][3]
Developer(s)
June 10 2024
Proprietary
Models[edit]
Apple Intelligence consists of an on-device model as well as a cloud model running on servers using Apple silicon. Both models consist of a generic foundation model, as well as multiple adapter models that are more specialized to particular tasks like text summarization and tone adjustment.[4]
According to a human evaluation done by Apple's machine learning division, the on-device foundation model beat or tied equivalent small models by Mistral AI, Microsoft, and Google, while the server foundation models beat the performance of OpenAI's GPT-3, while roughly matching the performance of GPT-4.[5]
Apple's cloud models are built on a Private Cloud Compute platform, designed heavily with user privacy and end-to-end encryption in mind.[6]
Functionality and features[edit]
Writing tools[edit]
Apple Intelligence features writing tools that are powered by AI, including Rewrite and Proofread. These features help enhance users' writing to make it more friendly, concise or professional, similar to Grammarly's AI writing features. It can also be used to generate summaries, key points, tables, and lists from an article or piece of writing.[7][8] ChatGPT will be available as part of Writing Tools.[9]
Image Playground[edit]
Apple Intelligence can also be used to generate images on-device with the Image Playground app. Similarly to OpenAI's DALL-E, it can be used to generate images using AI, using phrases and descriptions to create an image with customizable styles such as Animation and Sketch. In Notes, users can access Image Playground on iPad through the Image Wand tool in the Apple Pencil palette without having to open the Image Playground app. Rough sketches can be transformed into images.
Genmoji[edit]
Using Apple Intelligence text-to-image models, users can create original Genmoji images by typing descriptions.[10] Users can pick people in photos and create Genmoji images that resemble them.[11] Similarly to emoji, Genmoji can be added inline to text messages and can be shared in Messages and in supported third-party apps as stickers or Tapback reactions.[12][13]
Siri overhaul[edit]
Siri, Apple's virtual assistant, has been updated with enhanced capabilities made possible by Apple Intelligence. The latest iteration features an updated user interface, improved natural language processing, and the option to interact via text by double tapping the home bar without enabling the feature in the Accessibility menu. In addition, Apple Intelligence adds the ability for Siri to use personal context from device activities to make conversations feel more natural and fluid.[9]
Photos[edit]
Apple's Photos app includes a feature to create custom memory movies and enhanced search capabilities. Users can describe a story , and using Apple Intelligence, selects matching photos and videos. It organizes these into a movie with a narrative arc based on identified themes. Additionally, users can search for specific photos or videos by description and/or keyword, and Apple Intelligence can pinpoint particular moments within video clips.[14]
Code completion[edit]
Apple Intelligence is integrated into Apple's Xcode IDE, starting with version 16. This includes a predictive code completion feature, similar to GitHub Copilot, that runs on-device, using a model specifically trained on the Swift language and APIs. A new Swift Assist chat feature, using cloud models, generates and edits Swift code based on a user's prompt.[15] The on-device code completion feature requires a Mac with at least 16GB of memory to run.[16]
ChatGPT integration[edit]
As a result of the company's partnership with OpenAI, Apple Intelligence also includes a system-wide integration with ChatGPT, allowing Siri to determine when to send certain complex user requests to ChatGPT. This system-wide integration is powered by GPT-4o.[9] ChatGPT integration is opt-in by default, with users being prompted before any data or photos are sent to ChatGPT servers and IP addresses being obscured when requests are sent to OpenAI's servers.[17] Using ChatGPT features are free for all users without needing to sign in, however paid subscribers can sign in to gain access to paid features systemwide. Apple plans to integrate other models such as Google's Gemini into the system in the future.[18]
Development[edit]
Background[edit]
In October 2015, Apple Inc. acquired Perceptio, an on-device artificial intelligence modeling company.[19] Following the acquisition, Apple engaged in efforts to ensure its artificial intelligence operations remained covert; according to University of California, Berkeley professor Trevor Darrell, the company's secrecy deterred graduate students.[20]