iOS 27’s Apple Intelligence 2.0: What Developers Need to Know

Apple’s AI story has been a roller‑coaster since the first “Apple Intelligence” preview two years ago. iOS 26 added a handful of machine‑learning tricks, but the real leap is slated for iOS 27, which Apple is branding as Apple Intelligence 2.0. The update is expected to ship with iOS 27 SDK (Xcode 15.4) in the fall and will be the first iOS release that tightly integrates Google’s Gemini models via the new Apple Foundation Models framework.

Below we outline the headline features, why they matter for developers on both iOS and Android, and how you can start preparing your code‑base today.

---

1. Siri gets a full AI overhaul

What’s changing?

• A stand‑alone Siri app that can be launched from the home screen or via a widget.
• A chat‑style UI powered by a Gemini‑based large language model (LLM) for contextual, conversational answers.
• Agentic actions – hundreds of new intents that apps can expose through the SiriKit‑compatible Intents API.
• On‑screen awareness – Siri can reference UI elements that are currently visible, enabling “point‑and‑talk” interactions.
• Multi‑action requests – users can chain commands (e.g., “Find a coffee shop, book a table, and add the address to my calendar”).

SDK impact

• The iOS 27 SDK introduces SiriFoundation – a new framework that replaces the legacy Intents and IntentsUI for most use‑cases.
• SiriKit extensions now require a minimum iOS 27 deployment target and must be compiled with Swift 6 (or later) to take advantage of the new CodableIntent protocol.
• Android developers will see a parallel rollout in Android 15 with the Google Assistant SDK gaining similar Gemini‑backed capabilities. Cross‑platform teams should abstract intent handling behind a shared interface.

Migration checklist

1. Add the iOS 27 SDK to your Xcode project (xcode-select --install if you haven’t upgraded to Xcode 15.4).
2. Replace any INIntent subclasses with CodableIntent conformances.
3. Update your Info.plist to include the new SiriFoundation usage description key (NSSiriFoundationUsageDescription).
4. Test the new multi‑action flow using the SiriSimulator tool bundled with Xcode 15.4.
5. Create a thin Android wrapper that maps your existing Intent handling to the new Google Assistant actions, keeping the business logic in a shared Kotlin‑Multiplatform module.

---

2. Gemini‑powered photo editing in the Photos app

Feature preview

• Extend – generate content beyond the original frame (e.g., expand a close‑up of a landmark to show surrounding scenery).
• Replace – swap objects or backgrounds with AI‑generated alternatives while preserving lighting and perspective.
• Style Transfer – apply artistic styles in real time, powered by the same foundation model that runs on the device.

SDK impact

• Apple introduces AppleFoundationModels, a thin wrapper around on‑device Gemini inference. The framework ships with a PhotoEditing module that exposes three new PHEditingOperation subclasses.
• All operations are GPU‑accelerated via Metal and require devices with the A16 Bionic or newer. The minimum OS version for these APIs is iOS 27.
• For Android, Google is releasing GeminiEdit in the Jetpack Media3 library, offering parity for cross‑platform image‑processing pipelines.

Migration checklist

1. Add AppleFoundationModels to your Podfile or Swift Package Manager manifest.
2. Guard the new editing calls with if #available(iOS 27, *) and fallback to Core Image filters on older versions.
3. Update your UI to show the new editing toolbar only on supported devices (UIDevice.current.hasA16Bionic).
4. Port any server‑side image generation to the on‑device model to reduce latency and preserve privacy.
5. Sync the Android implementation using the GeminiEdit library to keep feature parity.

---

3. AI‑generated custom wallpapers

How it works

Users can invoke a new Wallpaper Generator from Settings or via Siri: “Create a wallpaper with a sunrise over the mountains in pastel tones.” The request is sent to the on‑device Gemini model, which returns a 4K‑resolution image that can be saved directly to the Photos library.

SDK impact

• The WallpaperKit framework adds a generateWallpaper(prompt: String, completion:) API. It returns a UIImage and a PHAsset identifier.
• The API respects user privacy – all generation runs locally; no network traffic is required unless the user opts into the optional cloud‑enhanced mode.
• Android’s counterpart appears in androidx.walls as WallpaperGenerator.generate(prompt).

Migration checklist

1. Import WallpaperKit and request the new NSWallpaperGenerationUsageDescription permission.
2. Provide a UI that falls back to the classic wallpaper picker on iOS 26 and earlier.
3. Cache generated images using NSCache to avoid repeated inference on the same prompt.
4. Synchronize the prompt‑to‑image workflow with the Android WallpaperGenerator for a consistent cross‑platform experience.

---

4. Universal subtitles for any video

What users will see

A toggle in Settings → Accessibility → Subtitles will enable automatic speech‑to‑text captions for any video playback, including third‑party apps that use AVPlayer or AVKit.

SDK impact

• Apple adds a system‑wide subtitle overlay that apps can opt‑in to via the AVPlayerItem.isSubtitleEnabled property.
• The overlay is powered by the same on‑device Gemini model used for speech recognition, delivering near‑real‑time captions with a latency of ~150 ms.
• Android developers will receive a similar feature in Media3 via SubtitleGenerator.enableForAllPlayers().

Migration checklist

1. Update your video playback code to set isSubtitleEnabled = true when the user enables the global toggle.
2. Test with various audio languages; the model currently supports English, Spanish, Mandarin, Hindi, and French out of the box.
3. Provide a fallback UI for iOS 26 devices that still rely on external caption files.
4. Implement a shared subtitle service in your cross‑platform layer so the same settings propagate to Android.

---

5. Natural‑language shortcut creation

Feature overview

The Shortcuts app now accepts free‑form text to create new automations: “When I arrive at the office, set my status to ‘In a meeting’ and launch the Zoom app.” The request is parsed by the Gemini model, which then assembles the appropriate actions.

SDK impact

• A new ShortcutBuilder class in Intents lets developers expose custom actions that can be discovered by the natural‑language parser.
• Each action must conform to the LLMAction protocol and provide a short, human‑readable description.
• The Android side receives ShortcutComposer in the androidx.shortcut library.

Migration checklist

1. Define your app’s custom actions as LLMAction subclasses and register them with ShortcutBuilder.register(action:).
2. Add localized descriptions for each action to improve parsing accuracy.
3. Test the end‑to‑end flow using the new Shortcut Playground in Xcode 15.4.
4. Mirror the action registration in Android using ShortcutComposer.register(action) for cross‑platform parity.

---

Cross‑platform considerations

Feature	iOS SDK	Android SDK	Minimum hardware
Siri overhaul	SiriFoundation (iOS 27)	Google Assistant SDK (Android 15)	A16 Bionic / Snapdragon 8 Gen 3
Photo editing	AppleFoundationModels (iOS 27)	GeminiEdit (Jetpack Media3)	A16 Bionic / Snapdragon 8 Gen 3
Wallpaper generator	WallpaperKit (iOS 27)	androidx.walls (Android 15)	A16 Bionic / Snapdragon 8 Gen 3
Universal subtitles	AVPlayerItem.isSubtitleEnabled (iOS 27)	SubtitleGenerator (Media3)	Any device with Neural Engine / Tensor Cores
Natural‑language shortcuts	ShortcutBuilder (iOS 27)	ShortcutComposer (Android 15)	A16 Bionic / Snapdragon 8 Gen 3

Practical advice for mixed‑platform teams

1. Adopt a shared core – Use Kotlin‑Multiplatform or Swift‑Kotlin bridge libraries to house the business logic for intents, image processing, and subtitle handling.
2. Feature‑flag per OS version – Guard new APIs with runtime checks (if #available(iOS 27, *) and if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.UPSIDE_DOWN_CAKE)).
3. Automated UI tests – Leverage Xcode’s SiriSimulator and Android’s UiAutomator to verify multi‑action Siri/Assistant flows before release.
4. Privacy compliance – Both platforms run the Gemini models on‑device by default. Ensure you disclose any optional cloud‑enhanced processing in your privacy policy.

---

Preparing for the WWDC preview

Apple has promised a live demo of the new Siri UI and a hands‑on showcase of the Extend photo tool at WWDC. Developers should:

• Register for the Apple Developer Program if they haven’t already – the new SDKs are only available to members.
• Download the iOS 27 beta from the Apple Developer portal as soon as it appears (usually a week after the keynote).
• Review the updated Human Interface Guidelines for AI‑driven interactions; Apple stresses clear affordances and user‑controlled data flow.

---

Bottom line

iOS 27’s Apple Intelligence 2.0 is more than a marketing buzzword; it introduces concrete, on‑device AI capabilities that will reshape how apps interact with users. By upgrading to the iOS 27 SDK, embracing the new AppleFoundationModels and SiriFoundation frameworks, and aligning Android implementations with the parallel Gemini‑powered APIs, you can deliver a consistent, privacy‑first experience across both ecosystems.

---

What Apple Intelligence features are you most excited to integrate? Share your thoughts in the comments, and keep an eye on the upcoming WWDC session for a deeper dive.