Everything Apple Announced at WWDC 2024: AI Features, New Siri ...

Apple typically uses its annual developer conference to announce big software updates and introduce new devices. But this year, Apple set aside the gadget talk and left plenty of room for what everyone expected would be the main topic of WWDC: all of the shiny new AI features coming to iPhones, iPads, and Macs.

Of course, Apple is arriving late to the artificial intelligence party, and has found itself in the position of needing to partner with a more established AI company to gain a foothold in the current arms race. In April, rumors emerged that Apple might be partnering with Google to utilize its Gemini AI on iPhones, but that doesn’t seem to have panned out. Instead, Apple is partnering with OpenAI for its first big batch of AI enhancements.

Here’s everything Apple announced at WWDC. Most of these features will become available in the fall, when Apple typically updates the software that runs on each of its devices.

Apple’s head of software engineering, Craig Federighi, debuts Apple Intelligence.

In Apple’s world, AI doesn’t stand for artificial intelligence. It stands for Apple Intelligence.

In a series of prerecorded demos, Apple laid out a vision for how users will interact with these new AI features, keeping the focus on personalized tasks and underscoring the usefulness of the AI-assisted capabilities. Apple says it can use AI to provide a highly personalized experience for you, the iPhone user, because its system is able to collate all sorts of information the AI has learned about how you use your device.

Apple’s AI can help with writing emails, reports, and personal text messages. The AI can also generate images inside Messages that are informed by the context of the conversation you’re having. If you and your friends are chatting about throwing a rooftop party, it can generate an image of what that party might look like. It can even generate AI images of the people you’re talking to. So if you’re texting your mom, the AI can sync with any photos you have identified as showing her face and then use those to generate an illustration of her. (If your mom is anything like mine, she will deeply hate this.)

Apple Intelligence also has advanced writing tools that can generate full emails or messages, or review what you’ve written to change the tone of a work email to help you avoid saying something you might later regret.

The artificial intelligence tools will gain summarization features that can be used to give you the gist of a web article or catch you up if you’re dropping back into a particularly vocal group chat after some time away. Summarization is something the other big AI players have been doing for a while. In fact, Google’s AI overviews have caused a ruckus recently after its ability to summarize search results have been called out for being misleading—sometimes just outright wrong—and also have been demonstrated to siphon content from original creators.

Apple was also eager to point out that many of these AI capabilities are processed privately on the device itself, and that, only when necessary, it can use secure cloud-based AI processing to complete your requests.

ChatGPT integration.

Additionally, there are new capabilities being added to iOS that come courtesy of Apple’s partnership with OpenAI. The most visible is that some Siri requests and generative text requests can be handed off to ChatGPT—powered by GPT-4o, Apple says—to complete them. OpenAI CEO Sam Altman was even spotted at the Apple campus today, though he didn’t appear in Apple’s promo video. Read all about this new partnership in Will Knight’s WIRED story.

Apple Intelligence requires phones with the A17 chip or devices with at least M1 chips. That means it can run on the iPhone 15 Pro and Pro Max, the new iPhones coming out this fall, or any iPad or Mac with an M1 chip or later.

New Siri.

Siri, Apple’s longtime digital assistant that no one really likes, is getting an AI glow-up. Apple says Siri is now smarter and better able to determine intent and decipher more complicated information. You can ask it several questions in a row, and it’s smart enough to understand that the questions are related to one another—something known in the AI world as contextual awareness.

It can complete more tasks than before, making it more helpful than it was in the old “alarm-setting” era. The more powerful Siri experience will be incorporated into Apple’s core apps, and developers will be able to add the new Siri’s capabilities to their own apps.

You can type out requests to the new Siri, like asking it to set an alarm late at night without waking the person sleeping next to you. (Reminder: Google Assistant has been doing this for a while.)

Finally, Siri has a new look: a new infinity-symbol-like logo, and a new status alert that makes a colorful ring appear around your entire phone’s screen when the assistant is listening to your request or generating a response.

The new Photos app coming in iOS 18.

The Photos app will get some additional editing features that will let you do things like make more elaborate enhancements to existing photos and remove unwanted people or things from the frame. (These are all abilities that have already been in Pixel and Samsung phones.)

Clean up the beach.

Apple is also making photo organization cleaner. Its new Apple Intelligence features will enable it to better recognize people in your contacts and group them together so you can find photos easier. With Apple Intelligence, you can also search for photos with more natural search terms, like “pics of Gina riding a skateboard on the park bench.”

Genmoji in action.

Apple is using its newfound AI capabilities to create custom emoji on the fly. The new Genmoji lets you enter prompts to generate new emoji characters that match the mood of the message you’re writing. You can mix up classic emoji, or generate something new that’s more specific to your needs. Want an eggplant riding a cheeseburger? You can probably cook that up.

And yeah, it’s called Genmoji. It’s OK to laugh at it!

Make your home screen look however you want.

Many of the changes coming to iOS when an update arrives this fall have nothing to do with AI. The key theme at WWDC was “enhanced customizability,” and Apple showed off ways to better categorize your apps, arrange them on the screen to your liking, and control how apps are accessed. Yes, this means you can arrange your app icons however you like now, changing their color and moving them to any location on the iOS screen’s familiar grid. There’s also support for hiding apps in a locked apps folder that can only be accessed through Face ID verification, so you can keep all your X and OnlyFans activity away from prying eyes.

There are also helpful updates to Messages. A big one is the ability to schedule messages, a thing that Apple had somehow not supported before. You’ll also get the ability to style text, add fun text effects, and reply to messages with emoji Tapbacks.

Here's macOS Sequoia on three different Macs.

Apple’s new operating system for Mac is called macOS Sequoia. The stand-out feature among those demonstrated today is called iPhone Mirroring. It allows you to sync your iPhone with your Mac and control nearly everything on the handset from your Mac’s screen. You can make calls, work within apps, and interact with the phone’s home screen. Your phone remains locked while you do this.

The Safari browser will get an update in Sequoia. There’s a new Highlights feature in the browser, which can summarize the most pertinent information contained on a website or within search results. If you’re searching for movies or music, Highlights can instantly pop up with a preview of what you want to watch or listen to. The Highlights feature can also provide directions to the restaurant you’re searching for, or surface menus and reviews in an instant, things like that.

iPhone Mirroring

More support is coming for games and game developers, including an upcoming second version of Apple’s Game Porting Tookit and new Xcode support that lets developers make games that work across all of Apple’s devices and platforms.

A new iPadOS feature cleans up your handwriting and generates a font based on your personal scrawl so you can paste in text and make it look like you wrote it by hand.

The iPad is getting some interface updates that aim to make the user experience more intuitive. There’s a floating tab bar that hovers at the top of some apps and aims to stay away from everything you’re doing on the screen, and better customization options for your Control Center.

Calculator graphing.

But what people are most stoked about is that Apple is finally bringing its Calculator app to the iPad. A revamped version of the calculator app not only works like a calculator, but it has a new feature called Math Notes that lets you use the Apple Pencil (or your finger) to visually work out and solve math problems. It’s bound to be a godsend for kids looking to cheat on their math homework.

The watchOS operating system is gaining some new cycle-tracking and pregnancy-tracking features.

The Apple Watch will get an update that gives the device new health-tracking features comparable to what’s offered by dedicated sports watches like those from Garmin.

A new Training Load app analyzes how hard you exercise during workouts, and tracks your workout duration and how your effort changes over time. The new Vitals app passively monitors your health throughout the day and may even suggest you’ve been drinking too much.

The new Passwords app.

On the security front, Apple is also creating its own dedicated passwords app across its OS platforms. It works like many other password managers, in that it allows you to generate strong passwords for all your logins. It keeps all your credentials in order by storing logins, passwords, and verification codes, and by flagging potential password leaks and security breaches. It’s also available to Windows users on the iCloud for Windows app.

Apple’s payments app has a new feature that lets users transfer money far more easily than before. With Tap to Cash, you can initiate a mobile payment and then just hold two iPhones together to transfer the money from one user to another. It’ll make paying back friends for a round of drinks much easier. (Sorry Venmo, you had a good run.)

AirPods get gesture controls.

Apple’s in-ear headphones will gain new gesture controls that let you nod your head to answer an incoming phone call, or shake your head to decline it. They’re also getting better noise isolation and canceling capabilities, so you can have more easily understandable conversations at your local jackhammer conventions.

Apple will add a few software features to its streaming TV platform. There are some accessibility updates, like better subtitles that appear when you mute right when the dialog is spoken, or pop up if you rewind a segment to play it again.

There’s also a new “In Sights” banner that pops up when you pause what you’re watching. It shows you the names and faces of the onscreen actors, as well as the characters they’re playing. It can also identify any of the songs that are playing in the scene. (Amazon Prime Video does this too through a feature called X-Ray.)

Apple’s Vision Pro mixed-reality headset just came out in February, and Apple has already announced the first substantial update to the face computer’s operating system. The Vision Pro didn’t get great reviews at launch, and so visionOS needs some enhancements to keep pushing the conversation forward.

The biggest update for the VisionPro will be coming to Spatial Photos, which will include a feature that makes ordinary pictures look like Spatial photos by giving them stereoscopic depth and movement.

Spatial Photos

Apple is also enhancing spatial video, partnering with companies like Vimeo, which has a player available as a new app on the VisionPro. Still no YouTube though. (Apple says the Vision Pro can run 1.5 million compatible iPhone apps.)

Apple is adding better hand gesture recognition to visionOS, and offering something called Train Support, which is meant to make the Vision Pro work better for people while they’re traveling by rail.

In the coming months, Apple says it is also making Vision Pro available to more countries, like China, Japan, Australia, Canada, and the United Kingdom.